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ABSTRACT 



The test development plan described here is an outgrowth of 
a 3 -year California state project to address the English-as -a- Second-Language 
(ESL) placement needs of adult education agencies in the state. The test 
development plan provides specific directions for the development of 
operational instruments anchored to the state's standards, and includes a 
recommended placement process for the standards and general guidelines for 
whole test and reading, writing, and listening item development. An 
introductory section describes the placement process and guidelines, and 
outlines detailed reading, writing, and listening specifications. Appended 
materials include a list of working group members, criteria for estimating 
the range and discrimination point of prototype reading and listening items, 
examples of reading/writing text adaptation, sample responses for generate 
writing tasks, and rater training procedures for scoring generate writing 
tasks. Contains 7 references. (MSE) (Adjunct ERIC Clearinghouse on Literacy 
Education) 
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Introduction 



The test development plan described in this document is an 
outgrowth of a three-year effort supported by the California Department of 
Education (CDE) to address the English-as-a-Second-Language (ESL) 
placement needs of adult education agencies in the state. ^ The plan 
includes text and item specifications for reading, writing, and listening 
and is intended for use by a test development team to guide the production 
of instruments appropriate for placing students into levels defined by the 
California English-as-a-Second-Language Model Standards for Adult 
Education Programs (California Department of Education, 1992). 2 The 
work which led to this test development plan was carried out at the UCLA 
Center for the Study of Evaluation in collaboration with an ESL Assessment 
Working Group. ^ (See Appendix A for the list of working group members.) 
The final project reports for the three years of work (1992-1995) describe in 
detail the tasks undertaken each year (Butler, Weigle, & Sato, 1993; Weigle, 
Kahn, Butler, & Sato, 1994; Kahn, Butler, Weigle, & Sato, 1995). 

The test development plan provides specific direction for the 
development of operational instruments anchored to the Model Standards 
and includes a recommended placement process for use with the Model 
Standards and general guidelines for item and whole test development. 
The specifications for the development of reading, writing, and listening 
items follow in the colored pages. 



^Henceforth in this document, adult education agency or agencies in California will be 
referred to as “agency” or “agencies.” 

^Henceforth in this document, the English-as-a-Second-Language Model Standards for 
Adult Education Programs will be referred to as the Model Standards. There are seven 
proficiency levels designated in the Model Standards: beginning literacy, beginning low 
(BL), beginning high (BH), intermediate low (IL), intermediate high (IH), advanced low 
(AL), and advanced high (AH). The Adult ESL Assessment Project addresses placement 
only into levels beginning low through advanced high. 

^Henceforth in this document, the ESL Assessment Working Group will be referred to as 
the working group. The working group, which was supported by the CDE, consisted of one 
representative from each of thirteen agencies in California and played an active, vital role 
in the project work. (See Butler, Weigle, & Sato, 1993, pp. 3-4, for a detailed explanation of 
the role of the working group.) 



Placement Process for Model Standards Levels 



In order to develop item specifications that match not only the 
content of the Model Standards but the needs of adult education agencies, a 
placement process was drafted in collaboration with the working group. 
Two overriding issues in placement testing led to the development of the 
placement process; 1) the need for group-administered tests that could be 
easily scored and 2) the difficulty of attempting placement into six levels 
with only one instrument. The three-tiered process, presented 
schematically in Figure 1, addresses both issues and provides the 
operational context for the instruments that will be produced from the 
specifications in this document. Key features of the process are discussed 
below. 

An ini tial screen identifies beginning literacy and beginning low students. 

Most agencies already conduct some form of intake interview for 
administrative purposes and often use this process to identify those 
students with minimal or no literacy or oral skills. These students are 
usually placed immediately into beginning literacy or beginning low and no 
further testing is required. Inclusion of the intake interview as an initial 
screen serves to formalize its function in the overall placement process. 

A second screen directs students to either low- or high-level testing. 

The second screen is intended to be a quick procedure to make gross 
distinctions between lower and higher proficiency students. It will identify 
additional beginning low students, who will not be required to undergo 
further testing, and will direct all other students to appropriate low- or 
high-level placement tests. In order to accommodate varying agency 
needs, the second screen will be agency specific in terms of format and skill 
area focus. Some agencies may decide to use a group-administered 
screening test which could involve listening, reading, or writing, while 
others may prefer to make the second screen an extension of the intake 
interview (initial screen) by including a few oral questions, a short reading 
passage, or a simple writing task. 



Figure 1 

ESL Placement Process 
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*There are seven proficiency levels designated in the Model Standards: beginning hteracy, beginning low 
(BL), beginning high (BH), intermediate low (IL), intermediate high (IH), advanced low (AL), and 
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Final placement decisions are based on low- and high-level instruments. 

Since most beginning low students will be identified by the first or second 
screen, low-level instruments will be used primarily to place students into 
beginning high, intermediate low, and intermediate high. High-level 
instruments will place students into intermediate low, intermediate high, 
advanced low, and advanced high. An important feature of this process is 
that both low- and high-level instruments will allow for placement into the 
intermediate levels should the second screen fail to direct a student to the 
most appropriate level test. 

The specifications in this document are intended to guide the 
development of instruments for use at this final stage of placement. Low- 
level instruments are designated as Level A and high-level instruments as 
Level B. The Level A and Level B instruments will include some 
combination of listening, reading, and writing items, since these skill 
areas are most amenable to group testing. 

Guidelines for Test Development 

The test development process recommended here is iterative; the 
items and the texts on which they are based must be revisited at every stage 
as new data help clarify how the items are working with the target 
population. The key steps in the process are highlighted and then 
discussed. 

Assemble a test development team. 

This first step is necessary to ensure the development of quality 
instruments. A core team should be assembled which includes experts 
with experience in language testing, ESL instruction, and at least one 
person with psychometric expertise to analyze test results so that data can 
be effectively evaluated. The core team will oversee the test development 
process. Responsibilities will include completing the specifications, 
establishing and canying out a text and item review process, training item 
writers, small- and large-scale testing, data analysis, developing and 
implementing a plan for whole test construction, conducting validity 
studies, and monitoring cutoff scores for placement. 



Constitute an advisory committee. 

The core test development team should meet regularly with an advisory 
committee consisting of one or more language testing experts and several 
representatives from agencies in California who will administer the tests 
once they are available. Language testing experts will provide input on 
further development of the specifications, on the plan for whole test 
construction, and on issues of reliability and validity. Continued input 
from agency representatives will help verify the content and language 
appropriateness of texts and associated items and determine what is 
feasible operationally given the realities of individual agency situations. 



Review and complete the specifications in this document. 

This document includes specifications for developing reading, writing, and 
listening items appropriate for placing adult ESL students into Model 
Standards proficiency levels. The specifications are a result of the 
prototyping effort undertaken in the third year of project work‘d in which 
preliminary assessment models were developed for text types and skills 
abstracted from the Model Standards and identified by the working group 
as critical for placement. While the specifications are comprehensive in 
that item types were developed for most of the skills identified,® the item 
types presented here are by no means exhaustive. The core team is 
encouraged to experiment with additional item types that seem promising 
using new source material.® 



It is also important to note that the reading and writing specifications are 
more fully developed than the listening specifications. This is due in part to 
the reading and writing skills being more fully articulated than the 
listening skills in the Model Standards. With reading and writing, the 
move from the Model Standards to test items was direct and clear which 



^Prototyping refers to the development of preliminary assessment models that serve as 
examples for item writers. The prototyping process is iterative and involves items being 
drafted, tried out, reviewed, revised, and tried out again. Once the items have been tested 
on a large scale and have undergone final revision, they become part of the item 
specifications and serve as models for producing operational assessments. 

®Item types were not developed for the reading skill Make Inferences because the texts 
selected did not lend themselves to assessing this skill. 



®New item types must go through a prototyping process that parallels the one used in 
developing the prototype items in these specifications. (See Kahn et al., 1995, for a 
description of the process.) 
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facilitated the abstraction of skills and the definition of underlying 
constructs. It also aided in the verification process in which working 
group members matched proposed skill definitions to Model Standards 
objectives. With listening the move from the Model Standards was more 
complicated because of the difficulty in defining underlying constructs and 
verifying their match to the Model Standards. (See Kahn et al., 1995, for 
discussion.) In addition, small-scale tryouts with listening items could 
only be conducted on a limited basis given the inherent logistical difficulties 
in obtaining quality recordings of texts, associated items, and directions. 
Thus the combined lack of specificity in the Model Standards and the 
logistical complexities associated with testing listening impacted what 
could feasibly be accomplished. 

Because the specifications have not yet been used to generate new items, 
they must be carefully reviewed and, where necessary, completed by the 
core team. Once the existing specifications have been completed and 
reviewed, they should be used by the core team to produce initial 
operational items and a procedure for training outside item writers should 
be developed. It is important to emphasize that specifications are dynamic 
and must be periodically reviewed and revised as needed as feedback 
regarding their effectiveness is received from item writers. 

Identify and train item writers to use the completed specifications. 

Once a complete set of specifications is available, potential item writers can 
be identified and trained. Item writers should be familiar with the adult 
ESL population and with the Model Standards. Background in these two 
areas will facilitate the selection of appropriate material for text and item 
development. A training session should take potential item writers 
through the specifications and should cover such issues as 1) text selection, 
including appropriate and inappropriate sample texts, 2) procedures for 
adapting authentic texts and producing original texts if necessary, and 
3) item type appropriateness in relation to skill, text type, and level. 
Training should include practice drafting items and critiquing them as a 
group. Following the training, potential items writers should be asked to 
submit a sample set that includes a text and associated items for review by 
the core team to determine whether a person shows promise as an item 
writer. 

14 
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Identify potential texts, adapt if necessaiy, and submit for review. 



Core team members or item writers who select texts should indicate the 
intended target level of each text and the types of items that might 
accompany it. An internal review process must be established to assure 
that potential texts meet selection criteria. As part of that process, at least 
two members of the core team should independently review every text 
submitted for consideration. Approval forms should be developed to help 
systematize the review process. If there is disagreement about 
appropriateness, a third member of the core team should review the text. 
After a text has been approved internally, it should be given to at least two 
advisory committee members from adult agencies and an outside language 
testing expert for review. Once a text has received final approval, it goes to 
item writers for item development. 



Draft test items and submit for review. 

As many items as possible should be drafted for each approved text to allow 
for item loss as a result of poor performance during pre- and pilot testing. 
Items should be reviewed by the core team and one or two advisory 
committee members in a process that parallels the text review process. 
Once the items are approved for pretesting, item sets should be assembled 
for each text and appropriate directions drafted. 



Conduct initial small-scale tryouts. 

Initial small-scale tryouts at appropriate levels are critical to determining 
whether the items and directions as formulated are comprehensible for the 
test taker. This phase, referred to as pretesting, provides the first 
opportunity to assess the appropriateness of the text and associated items 
for the target levels with test takers. Pretesting also allows for an initial 
indication of the amount of time needed for students to work through a text 
and accompanying items and for the identification of problems such as 
wording, familiarity with response formats, and task clarity. With writing 
tasks, the pretesting phase also shows whether a given prompt will elicit a 
ratable sample. 



A number of texts and associated items can be pretested at the same time 
even though they may not be intended for the same form of a test. The goal 
of pretesting is to collect information at the task and item level that will 
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inform revisions prior to large-scale tryouts. With some texts and 
associated items, a single pretest administration will provide the necessary 
information for revision, with others it may be necessary to pretest and 
revise several times within operational parameters (e.g., cost issues and 
access to students). The intent is to be able to conduct large-scale tryouts 
with sets of items that are as strong as possible. 

Conduct large-scale testing. 

Items should be revised based on pretesting results and large-scale testing 
should be conducted. This phase, referred to as pilot testing, involves the 
administration of the items to a large number of students across an 
appropriate range of levels so that, in addition to revisiting content issues, 
statistical analyses can be run to determine whether the items are 
performing as expected for placement purposes. The data obtained from 
pilot testing efforts allow for further revision and refinement of items and 
provide further indications of the estimated range of the items as well as 
the points at which the items best discriminate.'^ 

Appendix B provides the criteria used in the prototyping effort to assess the 
estimated range and discrimination point for reading and listening items. 
These criteria serve as a point of departure for establishing criteria for 
operational tests. It will be important to remember that with operational 
tests, criteria must be carefully monitored over time to determine accuracy 
and effectiveness of application, particularly as agencies continue to fine- 
tune their alignment with the Model Standards. A process for aggregating 
data across agencies in California will be essential for assuring refinement 
of the criteria vis-a-vis the Model Standards levels. 

Develop a whole-test construction plan. 

The design of a whole test is influenced by two primary sets of 
issues; content considerations and operational constraints. Content 
considerations include skill area focus, sampling of skills, and variety of 
topic and item formats. Operational constraints include administration 
time, mode of delivery, and scoring procedures. 



^The estimated range indicates the proficiency levels at which an item seems most 
appropriate in terms of content and performance. The discrimination point indicates the 
two adjacent levels where an item seems to best differentiate student performance. 
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For this test development effort, the following whole-test decisions have 
already been made in conjunction with the working group regarding both 
content and operational issues. 

1. Both Level A and Level B instruments will assess test taker ability in 
three skill areas — reading, writing, and listening — with reading and 
writing in one section and listening in another. 

2. Placement decisions will be based on performance in all three skill 
areas, although tests will be constructed in such a way that separate 
scores can be reported for each area. 

3. Sampling of skills within a given skill area will be based on 
recommendations found in the item specifications for reading, writing, 
and listening. The recommendations are a result of several meetings 
with working group members in which they were asked to prioritize 
skills for placement by indicating if a given skill was essential or 
optional for Level A and Level B tests. This information is presented on 
the overview page for each skill under Test Level. 

Note: , The recommendations found in the listening item specifications 
are based on preliminary discussions with the working group and will 
need to be systematically reviewed by the advisory committee. 

4. There will be a variety of item formats (both constructed and selected 
response); however, an attempt will be made to limit the number of 
different formats within a given section of the test. Ease of scoring will 
be a primary consideration in assembling whole tests. Example items 
should be included as needed as well as warm-up items at the beginning 
of the listening section to orient the test taker to the aural modality. 
Neither example items nor warm-up items should be scored. 

Assemble and pilot whole tests. 

Assemble at least two forms of each test for pilot testing. Complete test 
forms must be pilot tested to determine how each instrument is performing 
as a whole test. Timing adjustments can be made if necessary to assure 
that examinees have ample time to complete the test. Acceptable levels of 
reliability must be established and initial cutoff ranges for placement 
estimated. In addition, further item-level analyses will guide revision. 



An operational test must be monitored regularly to determine if cutoff 
ranges are allowing for effective placement decisions vis-a-vis course 
content. Over time it should be possible to adjust cutoffs so that students 
are being placed into classes appropriate for their language ability. 

Establish validity of instruments. 

Establishing the validity of an instrument is a critical part of any test 
development effort. Validity deals with whether an instrument is 
measuring what it purports to measure and is critical to assuring that 
decisions made on the basis of test performance are fair and equitable. 
Validity is always addressed with reference to a specific context and group 
of test takers. Thus, a test that is valid in one situation may not be valid in 
another. 

There are several dimensions to validity and for any given test situation the 
dimensions which impact the overall validity of the instrument must be 
identified. In the current effort, construct, content, and face validity are all 
relevant. Construct validity relates to the effectiveness with which the 
language skill descriptions in the Model Standards have been captured by 
the specifications and the test items. Content validity deals with the extent 
to which the test items provide adequate and appropriate skill coverage to 
allow for an accurate assessment of student ability in reading, writing, and 
listening as described in the Model Standards. Face validity refers to 
perceptions on the part of test takers and score users about the match 
between test content and the skills and tasks emphasized in the Model 
Standards and taught in courses. Construct and content validity must be 
established for placement decisions to be accurate, and face validity must be 
assured so that practitioners and students feel confident about the 
placement decisions that are made. 

Initial steps taken towards establishing the validity of the instruments to be 
produced from the specifications in this document are described below and 
include recommendations for additional steps to be taken. 

1. Working group members began the validation process by systematically 
matching the categorization of text types and skills to the Model 
Standards skill area descriptors. Where there was lack of agreement, 
modifications were made to the text types and skills to more accurately 
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reflect the Model Standards content. Modifications underwent further 
review and consensus was reached. 

This process was carried out and completed for reading and writing text 
tj^es and skills, as well as listening text types. As discussed above, the 
investigation of the validity of listening skills proved to be more 
complicated. Due in part to time limitations, consensus was not 
reached, but the process led to several recommendations in the 
categorization and definition of listening skills which are reflected in 
the listening item specifications. To complete the process, however, 
these skills must be further reviewed and validated by the advisory 
committee. 

2. Working group members also prioritized skills for placement by 

determining whether a given skill is essential, optional, or 
inappropriate® on Level A and Level B tests based on the following 
primary considerations: (1) whether the skill is essential to 

determining a test taker’s ability in a given skill area and (2) whether 
the skill can feasibly be assessed on a placement test given limited 
testing time and other operational constraints. 

Item specifications were drafted for those skills identified as either 
essential or optional across all three skill areas, although skill 
prioritization for listening will need to be reviewed and validated by the 
advisory committee. A whole-test construction plan must assure 
adequate sampling across and within skill areas and the plan must be 
verified by the advisory committee. 

3. All prototype texts and items were reviewed both internally and 
externally to assure text appropriateness and quality of items. In 
addition, prototypes were pretested and pilot tested at several agencies 
statewide and qualitative data in the form of questionnaires, interviews, 
and classroom observations were collected from teachers, students, and 
administrators. Information gleaned from the qualitative data ensured 
that the texts and tasks were perceived as comprehensible, useful, and 

^Essential skills are critical for placement and must be assessed on every form of the test. 
Optional skills are important but not critical for placement and need not be assessed on 
every form of the test. Inappropriate skills are not critical for placement and should not be 
assessed on any form of the test. 



indicative of test taker ability. The process of obtaining input and 
feedback from these constituencies must continue as test items are 
developed. 

4. Text and item specifications were reviewed by a language testing 
specialist to ensure overall clarity and coherence and to make sure each 
item type is appropriate for the skill being tested. These specifications 
must now be verified by the core test development team and the advisory 
committee. 

5. As part of the item development process, various formats were pretested 
to control for the effect of testing method and determine the formats that 
are most effective for assessing specific skills. Item Type Notes provide 
details of specific instances. 

Although validity is addressed in the final guideline of this section, it is 
important to stress that establishing validity is an ongoing process that 
begins in the initial stages and continues throughout the development of 
operational instruments. It will take the combined efforts of the core team 
and the advisory committee to assure the validity of the instruments to be 
produced. 

What follows are the text and item specifications for reading, 
writing, and listening. Reading and writing tasks are integrated in that 
both are developed around the same source materials and are therefore 
presented together in the specifications. Listening is presented separately. 
A glossary of selected terms is included at the end of the specifications. 
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Beading and Writing Specifications 



Reading/writing specifications 



General overview 



General Overview 



The reading and writing specifications are divided into three sections: 

• Reading/writing text specifications 

• Reading item specifications 

• Writing task specifications 

Each section is organized as follows. First, there is an overview page 
which presents the general organization and defines critical terms used in 
the section. Then, specifications are presented including prototypes that 
exemplify the texts, items, or tasks being defined. A brief description of 
each section is provided below. 

Reading/writing text specifications 

Reading/writing text specifications include general guidelines for text 
selection and adaptation and specifications for the following text types 
abstracted from the Model Standards and validated for placement: 
Display, Paragraph, and Multi-paragraph Passage. 

Reading item specifications 

Reading item specifications are organized by skill. The following 
reading skills were abstracted from the Model Standards and validated 
for placement: Locate Specific Information, Draw Meaning, Extract 

and Combine Information, Interpret Relationships, and Make 
Inferences. For each skill, multiple item types are specified. 

Writing task specifications 

Writing task specifications are also organized by skill. The skill 
Generate was abstracted from the Model Standards and validated for 
placement.® Four task types are specified for this skill and a description 
of holistic scoring procedures and rubrics is provided. 



®The writing skills Copy and Complete were also abstracted from the Model Standards and 
validated for placement. However, it was decided in collaboration with the working group 
that it would be best to assess these skills at either the first or second tier of the placement 
process. Thus specifications for Copy and Coiftplete are not included in this document. 



Reading/writing text specifications 



Overview 



Reading/Writing Text Specifications 
Overview 



Text specifications are provided for the three reading/writing text types 
below. 

• Display 

• Paragraph 

• Multi-paragraph passage 

The text specifications are organized as follows. First, general guidelines 
for reading/writing text selection and adaptation are presented. Then, 
specifications for each text t}T5e are provided. The specifications include the 
following: 

General description 

A brief definition of the text t}T5e 

Length 

The recommended number of words 
Number of items per text 

The recommended number of test items associated with a text in an 
operational form of a test 

Note: For a given text, as many items as possible should be drafted and 
pretested to generate a sufficient number of acceptable items for an 
operational form of the test. 

Target level 

The Model Standards proficiency levels at which a text is targeted 

The target level of a text depends both on the text type and on the 
language in the text that must be interpreted to answer specific items. 
In order to assess a range of proficiency levels with a given text, the text 
should include vocabulary and structures targeted at several levels. 

Language is defined as targeted at a given level when test takers at that 
level are presumed to be able to interpret the language, while test takers 
at the next lower level are not. For guidance on appropriate language 
forms for the proficiency levels, see Sentence Types and Verb Tenses, 
Model Standards, pp. 74-77. 
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Reading/writing text specifications 



Overview 



•f-'X, 



Test level 

An indication of whether the text type is appropriate for Level A tests or 
Level B tests 

SkiUs 

The reading skills that can be tested using the text type 
Prototype text 

A text that exemplifies the text type 

All prototype texts were pilot tested in the form in which they are 
presented and were found to be promising as models for placement test 
development based on content considerations and feedback from 
teachers and students. 

Prototype text notes 

Explanatory notes about the prototype text that include a discussion of 
the text vis-a-vis the specified text attributes 



Tlmoughout the reading and writing specifications, texts are referred to as 
primary or secondary. These terms are defined below. 

Primary text 

The main text on which most reading items and writing tasks are based 

A primary text is either a display, paragraph, or multi-paragraph 
passage (defined below). 

Secondary text 

A supplementary text related to the primary text and critical to some 
reading and writing task types 

A secondary text is either an additional display or paragraph or a 
statement in the item stem which sets up a condition or a problem. 








Reading/writing text specifications 



Selection and adaptation 



Text Selection catd Adaptation 



Reading/writing texts are either selected from existing materials and used 
in their original form or adapted from existing materials and reproduced. 
(See Appendix C for examples of original and adapted texts.) What follows 
are general guidelines that should be adhered to in producing all 
reading/writing texts. 

1. All reading/writing texts should be selected from general content areas 
that are familiar to adult ESL students given their goals and 
experiences. General topics include but are not limited to shopping, 
banking, housing, health, transportation, current events, and 
community resources. Some common vocational topics such as 
employment and general workplace safety may also be appropriate. 

2. Care must be taken to select topics that will not quickly become dated or 
one that critically depends on background knowledge of specific people 
or events. 

3. The appropriateness of a text for a given level depends in part on factors 
such as familiarity of topic, visual aids, and complexity of content, 
vocabulary, or syntax, as specified in the Model Standards. All texts 
must be reviewed by content-area experts familiar with the Model 
Standards and the adult ESL population to evaluate their 
appropriateness for use at specific proficiency levels. 

4. Adapted texts should be based on materials originally written for a 
general English-speaking audience and not for language learning or 
assessment purposes. A text should remain as close to its original 
format as possible, although the following adaptations may be made: 

(a) Brand names, other proper nouns, addresses, and telephone 
numbers should be changed to avoid association with actual people, 
organizations, places, and so on. 

(b) A text may be shortened or edited for simplicity and clarity, or to 
make a given item type possible. On a Level A text, the language 
may be simplified in places, as long as connected discourse retains a 
natural flow. 

(c) Visuals may be added to help orient the test taker to the text. 
Existing visuals may be modified for clarity and simplicity. 

5. Permission must be obtained to use a text, whether adapted or in 
original form, on an operational test. The source of the text must be 
acknowledged wherever it appears on all test forms. 



Reading/writing text specifications 



Display 



Display 



General description 

A display is a text consisting of pieces of related information presented 
in words, phrases, sentences, and/or non-alphabetic forms (i.e., 
numerals or symbols) that are not connected into a single piece of 
discourse. A display may also contain visuals that orient the test taker 
to the content. Examples of displays include lists, menus, directories, 
indices, calendars, schedules, signs, labels, advertisements, forms, and 
tables. 

Length 

Between 20 and 75 words 
Target level 

Beginning low through intermediate high 
Test level 

Appropriate for both Level A and Level B tests 

Number of items per text 
Between two and six 

Skills 

Locate specific information 

Extract and combine information (in conjunction with a secondary text) 
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Reading/writing text specifications 



Display 



Prototype text 



Village Bike Shop 

World Class Bicycles 
SHIP Call! 257"2453 

WORLDWIDE 

854 Ocean Blvd • Palm Village 

• Bicycles for the entire family 

• Free lifetime service plan with purchase of any bike 

• All makes & models expertly serviced 

• Quality selection of accessories & apparel 

• Bicycle & exerciser rentals available 




Free Parking Available 
★ OPEN 7 DAYS ★ 




adapted from Donnelly Yellow Pages , Los Angeles, 1992 



Prototype text notes 

The display is an advertisement containing 47 words and eight pieces of 
non-alphabetic information (numbers and S 3 onbols). The example text 
is targeted primarily at the beginning levels, with some vocabulary 
targeted at the intermediate levels, and is thus appropriate for Level A 
tests. 
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Reading/writing text specifications 



Paragraph 



Paragraph 



General description 

A paragraph is a series of related sentences (at least two, usually no 
more than five). Examples of paragraphs include notes or messages, 
brief public information notices, instructions, and short newspaper or 
magazine articles. 

Length 

Between 40 and 100 words 

Target level 

Beginning high through advanced high 

Test level 

Appropriate for both Level A and Level B tests 

Number of items per text 
Between two and four 

Skills 

Draw meaning 

Extract and combine information 
Interpret relationships 
cause/effect 
compare/contrast 
generalization/example 
main idea/supporting details 
sequence of events 
Make inferences 
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Reading/writing text specifications 



Paragraph 



Prototype text 



Strange True Stories 

Police arrested James Haskell, 
seventeen, in March after he tried to 
buy beer at Wally’s Mini-Mart by 
using a stolen driver’s license — that 
of Douglas Cleaver. The man 
behind the counter at Wally’s knew 
it was stolen because he is Douglas 
Cleaver, who had had his license 
taken in a truck break-in two 
months earlier. Haskell fled when 
he heard Cleaver talking to the 
police but left his wallet, which 
contained his real driver’s license. 

adapted from The Los Angeles Reader, June 2, 1994 



Prototype text notes 

The text is a short magazine article consisting of an 80-word paragraph 
that narrates a series of events. The example text is targeted primarily 
at the intermediate levels and is thus appropriate for both Level A and 
Level B tests. 
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Reading/writing text specifications 



Multi-paragraph passage 



Multi-Paragraph Passage 



General description 

A passage is a series of related paragraphs (at least two, usually no 
more than ten). Examples of passages include public information 
notices, letters, and newspaper or magazine articles. 

Length 

Between 100 and 375 words 

Target level 

Beginning high through advanced high 

Test level 

Appropriate for Level B tests only 

Number of items per text 
Between four and eight 

Skills 

Draw , meaning J 

Extract and combine information 
Interpret relationships 
cause/effect 
compare/contrast 
generalization/example 
main idea/supporting details 
sequence of events 
Make inferences 



Reading/writing text specifications 



Multi-paragraph passage 



Prototype text 



City News 




City News' Person of the 
Week is adult school student 
Oscar Rodriguez, whose dream 
came true last week when he 
received his high school 
diploma after six years of study 
at City Adult School. 

Rodriguez said his primary 
motivation for graduating was 
his 17-year-old daughter, who 
becomes a high school senior 
this fall. “I wanted to have my 
diploma first,” the 35-year-old 
student explained. 

Rodriguez, who attended 
classes at the adult school to 



learn English and become a U.S. 
citizen, accomplished two feats 
in one year: getting his high 
school diploma and being named 
Citizen of the Year by the State 
Consortium for Adult Education. 

Rodriguez came to the United 
States 14 years ago from 
Guatemala. At that time, he felt 
no immediate need to learn 
English since the people he 
spent time with spoke primarily 
Spanish and he listened to 
Spanish radio and TV. 

Later, he realized he needed to 
learn English in order to get his 
high school diploma. So, in 
1988 he enrolled in City Adult 
School where he proved to be a 
very diligent and focused 
student. He found time to study 
and participate in community 
activities while working two 
jobs and raising a family. 

According to Consortium 
president Gerry Delaney, “The 
adult school is full of students 
like Rodriguez who have many 
responsibilities, but Rodriguez 



Thursday, June 23, 1994 

stands out because he always 
gets the job done with a smile and 
in a timely fashion.” The 
Consortium acknowledged 
Rodriguez* accomplishments by 
giving him the Citizen of the Year 
award. 

“I feel proud. I wanted to do 
this as an example for my kids,” 
said Rodriguez, who has another 
daughter, age 7, and two sons, 15 
and 10. 

In fact, Rodriguez has set a 
good example not only at school 
but at work. Last year he was 
named Most Outstanding 
Employee at County Hospital 
where he works as a nurses* aide. 

Receiving his diploma is only 
the first step to other ambitions 
for Rodriguez, who plans to 
continue his work in the medical 
field. “Now I want to go to 
college so I can become a nurse.” 

City News congratulates Oscar 
Rodriguez and wishes him 
continued success. 



from Culver City News^ June 23, 1994 



Prototype text notes 

The text is a newspaper article consisting of 10 paragraphs (367 words) 
that tells about a local current event. The example text is targeted 
primarily at the advanced levels, with some vocabulary and structures 
targeted at the intermediate levels, and is thus appropriate for Level B 
tests. 
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Reading item specifications 



Overview 



Reading Item Specifications 
Overview 



Item specifications are provided for the reading skills below. 

• Locate specific information (LS) 

• Draw meaning (DM) 

• Extract and combine information (EC) 

• Interpret relationships (IR) 

• Make inferences (MI)io 

For each skill, an introductory page provides the following information: 
General description 

A brief definition of the skill as operationalized in item types 
Text type 

The primary text (display, paragraph, or multi-paragraph passage) on 
which reading items are based 

Target level 

The Model Standards proficiency levels at which items testing the skill 
are targeted 

An item is defined as being targeted at a given level when test takers at 
that level are presumed to be able to answer the item correctly while test 
takers at the next lower level are not. 

The target level of an item is dependent on both the skill being tested and 
the language that must be interpreted (in the item stem, the primary 
text, or the secondary text) in order to answer the item. 




item types were not developed as part of the prototyping effort. Texts that lend 
themselves to assessing this skill need to be selected and item types developed following the 
prototyping process described in Kahn et al. (1995). 
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Reading item specifications 



Overview 



Test level 

An indication of whether the skill is essential or optional for placement 
on Level A tests and Level B tests 

Item types 

A brief description of each item type testing the skill 

The introductory page is followed by specifications for each item type 
associated with that skill. The specifications include the following 
information: 

Item format 

The format of the item type (multiple choice, select number correct, 
sequencing, unique answer) 

Stimulus attributes 

The characteristics of the input to the test taker, which include: 

(a) specific information about the primary text and, where appropriate, 
the secondary text 

(b) a detailed description of the item stem 

(c) an indication of where target-level language is found in the item 

Response attributes 

The characteristics of the response 

For constructed response items, a detailed description of the expected 
response is provided. 

For selected response items, the following are specified: 

(a) number of response options 

(b) characteristics of the key (correct option) 

(c) characteristics of the distractors (incorrect options) 

(d) language characteristics of the response options 
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Reading item specifications 



Overview 



Prototype item 

An item that exemplifies the item t3T)e 

Unless otherwise noted, all prototype items were pilot tested in the form 
in which they are presented and were found to be promising as models 
for placement test development based on statistical performance and 
content considerations. 

The key is indicated by an asterisk (*) for selected response items. 
Prototype item notes 

Explanatory notes about the prototype item include the following: 

(a) a discussion of the item vis-a-vis the specified stimulus and 
response attributes, including the target-level language 

(b) examples of acceptable and unacceptable responses for constructed 
response items 

(c) the estimated range and discrimination point for the item based on 
pilot testing results 

The estimated range indicates the proficiency levels at which the 
item seems most appropriate in terms of content and performance. 
The discrimination point indicates the two adjacent levels where the 
item seems to best differentiate student performance. (See Appendix 
B for criteria used to determine both estimated range and 
dis erimination p oint . ) 

Note: Some prototype items were modified based on pilot testing 
results. For these items, the estimated range and discrimination 
point are not available because they have not been pilot tested in their 
current form. 

Additional notes about the item t3Ties are provided at the end of each skill 
section. These notes include supplementary information about the item 
types from pre- and pilot testing, such as considerations for alternative 
item formats and variations in wording. 
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Reading item specifications 



Locate specific information 



Locate Specific Information (LS) 



General description 

The test taker will find specific information in a display. The 
information may be either non-alphabetic (numerals or symbols) or 
alphabetic (words, phrases, or abbreviations). The correct response to a 
question may be the specific information itself or a word or phrase from 
the display associated with the information. 

Text type 
Display 

Target level 

Beginning low through intermediate high 

Test level 

Level A: essential 

Level B: optional, as a warm-up 

Item types 

In item type LSI, the test taker locates and writes down specific 
information from a text. In item type LS2, the test taker locates specific 
information associated with a particular referent and chooses the 
correct referent from several options. 



Reading item specifications 



Locate specific information 



Item Type LSI 



Item format 

Unique answer 

Stimulus cUtrihutes 

The primary text is a display. 

The item stem consists of a question asking for a piece of alphabetic or 
non-alphabetic information found in the display. The question 
ordinarily begins with who, what, when, where, how much, or how 
many. 

Target-level language is found in either the relevant part of the display 
or the item stem, or both. 

Response cUtrihutes 

The test taker must write down the correct answer, which consists of a 
number, word, or phrase taken directly from the text. The test taker is 
not penalized for minor transpositions of numbers, misspellings, or 
grammatical errors as long as the meaning is clear. 
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Reading item specifications 



Locate specific information 



Prototype item 

Read the bicycle shop advertisement below and answer the question. 






BIKES GALORE! 



• Expert repairs — overnight service, most brands 

• Affordable rentals — hourly and daily rates 

• Lowest prices in town 

• Alt cycling accessories and parts 

• Exercise equipment ^ 

• Insure your bicycle. Ask about our anti-theft policy. 



( 3 % 

( 3 ^ 



- 2 WESTSIDE LOCATIONS - 



BIKES GALORE 
of Southdale 
9000 Western Ave. 

456-0915 

FREE PARKING IN REAR 



VISA 



DISCOVm 



BIKES GALORE 
of North Harbor 
3900 Vemer Blvd. 
278-1685 



HOURS: M-F 10-7 SAT 9-5:30 



adapted from Donnelly Yellow Pages , Los Angeles, 1992 



What time does Bikes Galore 
close on Wednesdays? 



Prototype item notes 

Prototype LSI requires the test taker to interpret the phrase “close on 
Wednesdays” in the item stem and the abbreviation “M-F” in the text and 
write down the correct time. Example responses appear below. 

Acceptable: 7:00; 7; seben (minor spelling error); 7 p.m.; Bikes 

Galore close at 7 (minor grammatical error) 

Unacceptable: 10-7 (does not demonstrate comprehension of the 

word “close”) 10; 5:30 



Estimated range: BL-IL 



Discrimination point: BH/IL 



Reading item specifications 



Locate specific information 




Item Type LS2 



Item format 

Multiple choice 

Stimulus attributes 

The primary text is a display containing at least three members of a 
referent category (e.g., names, places) that can be distinguished from 
each other through their association with non-alphabetic information in 
the text. Note that it is possible to use a display containing only two 
members of the category, in which case “both” or “neither” can be seen 
as additional members. 

The item stem consists of a question asking for a unique referent such 
as a person or place found in the text. The question ordinarily begins 
with who, what, where, or which and provides information that 
uniquely identifies the referent from other possible responses. 

Target-level language is found in either the relevant part of the display 
or the item stem, or both. 

Response attributes 

A set of three or four response options in the form of a word or phrase 
follows the stem. The test taker must circle the correct answer. The key 
is the correct referent. Distractors consist of other members of the same 
category (i.e., other persons or places) found in the text. 
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Reading item specifications 



Locate specific information 



( i 



Prototype item 



Here is an Apartment Guide that gives information about four 
apartments. Use the information to answer the question. 



Apartment Guide 



Apt. 

Features 


Willow 

Court 


Village 

Place 


Hamill 

Towers 


Pierce 

Plaza 


rent 


$750 


$550 


$700 


$950 


bdrms. 


2 


1 


2 


2 


baths 


2 


1 


1 


2 


parking 




♦ 


♦ 


♦ 


security bldg. 


♦ 




♦ 


♦ 


pool 


♦ 


♦ 






laundry 


♦ 




♦ 


♦ 


pets ok 




♦ 


♦ 




unique feature(s) 


• on bus line 


• all new 
appliances 


• all utilities 
paid 


• A/C 

• cable ready 



adapted from The Southern California Apartment Magazine, April 20-May 3, 1994 



Circle the correct answer. 

Which is the least expensive apartment? 

Willow Court *Village Place Hamill Towers Pierce Plaza 



Prototype item notes 

In Prototype LS2, the test taker must locate the amount of rent for each 
apartment and identify which is the least expensive. The item stem 
contains the phrase “least expensive,” which is appropriate for 
intermediate levels. 



Estimated range: IL-AH 



Discrimination point: IL/IH 



Reading item specifications 



Locate specific information 



LS Notes 



1. Item type LSI could be constructed in multiple-choice format if there 
were enough pieces of specific information in the display to provide 
plausible distractors; that is, information of the same category (e.g., 
times in Prototype LSI above) as the key. 

2. Item type LS2 could be constructed in unique-answer format; however, 
results from pretesting this item type indicated that the plausible 
answers are restricted to the category members (e.g., “Willow Court” 
and “Village Place” in Protot 5 q)e LS2 above) and it is much faster for the 
test taker to circle an answer rather than write it in a blank. 

3. Prototype LS2 was pretested with both “most expensive” and “least 
expensive.” Results indicated that the item with “most expensive” 
discriminated between beginning low and beginning high, while the 
item with “least expensive” discriminated from intermediate low 
through advanced high. The phrase “most expensive” may be more 
appropriate for beginning levels because the test taker does not 
necessarily need to understand the superlative in order to find the 
correct response. This is an example of how modifying the language 
can affect the difficulty of an item. 
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Reading item specifications 



Draw meaning 



' ) 



Draw Meaning (DM) 



General description 

The test taker will demonstrate factual understanding of one or more 
propositions in a paragraph or multi-paragraph passage. In 
answering this type of item, the test taker cannot simply glance through 
a text or match words or phrases to find a piece of information. Rather, 
the test taker must draw meaning from the text by interpreting one or 
more of the following: 

• vocabulary and sentence structure 

• cohesive devices, including pronoun reference, ellipses, and 
lexical cohesion 



Text type 

Paragraph or passage 
Target level 

Beginning low through advanced high 

Test level 

Level A: essential 
Level B: essential 

Item types 

In item type DM1, the test taker writes down a short answer to a factual 
question about a text. In item type DM2, the test taker selects one of four 
responses to a factual question about a text. In item type DM3, the test 
taker selects three of five responses to a factual question about a text. 
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Reading item specifications 



Draw meaning 



Item Type DM1 



Item format 

Unique answer 



Stimulus attributes 

The primary text is a paragraph or multi-paragraph passage. 

The item stem consists of a factual question about the text that can be 
answered with a word or a phrase. The question ordinarily begins with 
who, what, where, or when. 

The relevant part of the primary text contains target-level language. 
The item stem contains language at a lower level. 



Response attributes 

The test taker must write down the correct answer. A complete 
sentence is not required. The test taker is not penalized for misspellings 
or grammatical errors as long as the meaning is clear. 




42 



38 



\ 

./ 



Reading item specifications 



Draw meaning 



Prototype item 

Read the announcement below and answer the question. 



The Department of Parks and Recreation 
is sponsoring free Saturday evening concerts 

July 9, 16 and 23. 

Concerts will begin at 7 p.m. and will last about two hours. 
They will be held in the new outdoor amphitheater. 
Call 555-8632 for more information. 



adapted from Elk Grove Adult School ESL Placement Test 



Where will the concerts take place? 



Prototype item notes 

Prototype DM1 requires the test taker to interpret the phrase “be held” in 
the text and know that it is synonymous with “take place” in the item 
stem. Comprehension of the word “sponsoring” can also help the test 
taker eliminate “Department of Parks and Recreation” as a possible 
response. Example responses appear below. 

Acceptable: amphitheater; new outdoor amphitheater; 

amphither (i.e., minor misspelling) 

Unacceptable: Department of Parks and Recreation; outdoor; They 
will be held; July 9, 16 and 23 

Estimated range: BL-IH Discrimination point: IL/IH 
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Reading item specifications 



Draw meaning 






Item Type DM2 

Item format 

Multiple choice 

Stimulus attributes 

The primary text is a paragraph or multi-paragraph passage. 

The item stem consists of a factual question about the text. The correct 
answer to the question is ordinarily contained within a few adjacent 
clauses in the text, although clues to the correct answer may be found in 
other places in the text as well. 

The relevant part of the primary text contains target-level language. 
The item stem contains language at a lower level. 

Response attributes 

A set of four single-phrase or single-sentence response options follows 
each stem. The response options contain language at a lower level than 
the target level. The test taker must circle the letter of the correct option. 
The key is a phrase or sentence answering the question using words 
other than those in the text. 

The three distractors are grammatically accurate and plausibly relate to 
the content of the text, either by repeating or paraphrasing segments of 
the text or by building upon a word or idea found in the text (Popham, 
1990 ). 

At least one of the distractors incorporates words or phrases from the 
section of the text where the correct answer is found but with a change 
of meaning based either on alternative senses of words or on alternative 
interpretations of syntax. This type of distractor is intended to draw 
students who are matching words and phrases rather than reading for 
understanding. 

Other distractors are inaccurate in that they are either unsupported or 
contradicted by information in the text. 
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Reading item specifications 



Draw meaning 



Prototype item 



City News 




City News' Person of the 
Week is adult school student 
Oscar Rodriguez, whose dream 
came true last week when he 
received his high school 
diploma after six years of study 
at City Adult School. 

Rodriguez said his primary 
motivation for graduating was 
his 17-year-old daughter, who 
becomes a high school senior 
this fall. “1 wanted to have my 
diploma first,” the 35-year-old 
student explained. 

Rodriguez, who attended 
classes at the adult school to 



learn English and become a U.S. 
citizen, accomplished two feats 
in one year: getting his high 
school diploma and being named 
Citizen of the Year by the State 
Consortium for Adult Education. 

Rodriguez came to the United 
States 14 years ago from 
Guatemala. At that time, he felt 
no immediate need to learn 
English since the people he 
spent time with spoke primarily 
Spanish and he listened to 
Spanish radio and TV. 

Later, he realized he needed to 
learn English in order to get his 
high school diploma. So, in 
1988 he enrolled in City Adult 
School where he proved to be a 
very diligent and focused 
student. He found time to study 
and participate in community 
activities while working two 
jobs and raising a family. 

According to Consortium 
president Gerry Delaney, “The 
adult school is full of students 
like Rodriguez who have many 
responsibilities, but Rodriguez 
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stands out because he always 
gets the job done with a smile and 
in a timely fashion.” The 
Consortium acknowledged 
Rodriguez’ accomplishments by 
giving him the Citizen of the Year 
award. 

“1 feel proud. I wanted to do 
this as an example for my kids,” 
said Rodriguez, who has another 
daughter, age 7, and two sons, 15 
and 10. 

In fact, Rodriguez has set a 
good example not only at school 
but at work. Last year he was 
named Most Outstanding 
Employee at County Hospital 
where he works as a nurses’ aide. 

Receiving his diploma is only 
the first step to other ambitions 
for Rodriguez, who plans to 
continue his work in the medical 
field. “Now I want to go to 
college so 1 can become a nurse.” 

City News congratulates Oscar 
Rodriguez and wishes him 
continued success. 



adapted from Culver City News, June 23, 1994 



Circle the best answer for the following question. 

According to the article, which statement best describes many students 
at City Adult School? 

a. They are full-time students. 

b. They like Rodriguez because he is nice. 

*c. They make time for school even though they have a lot to do. 
d. They are always in a good mood and get their work done quickly. 



Prototype item notes 

Option C, the key, is a paraphrase of the relevant proposition in the text 
describing students at City Adult School. Option A incorporates the 
phrase “full-time students,” which is similar in form but not in 
meaning to “full of students” found in the text. Option B incorporates 
the exact words “like Rodriguez” found in the text but uses a different 
syntactic structure which changes the meaning of the phrase. Option D 
is a paraphrase of an adjacent clause and is inaccurate in that the 
clause refers to Rodriguez, not to the other students at City Adult School. 

Estimated range: IL-AH Discrimination point: IL/IH 
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Reading item specifications 



Draw meaning 



Item Type DM3 



Item format 

Select three correct 

Stimulus attributes 

The primary text is a paragraph or a passage. 

The item stem consists of a factual question about the text requiring the 
test taker to identify three correct answers. 

The relevant part of the primary text contains target-level language. 
The item stem contains language at a lower level. 

Response attributes 

A set of five single-phrase or single-sentence response options follows 
each stem. The response options contain language at a lower level than 
the target level. Each option is preceded by a blank. The test taker must 
check the three correct options. The keys are phrases or statements 
answering the question using words other than those in the text. 

The two distractors are grammatically accurate and plausibly relate to 
the content of the text, either by repeating or paraphrasing segments of 
the text or by building upon a word or idea found in the text (Popham, 
1990). The distractors are inappropriate in that they are either 
unsupported or contradicted by information in the text. 



Reading item specifications 



Draw meaning 



Prototype item 



City News 




City News' Person of the 
Week is adult school student 
Oscar Rodriguez, whose dream 
came true last week when he 
received his high school 
diploma after six years of study 
at City Adult School. 

Rodriguez said his primary 
motivation for graduating was 
his 17-year-old daughter, who 
becomes a high school senior 
this fall. “I wanted to have my 
diploma first,” the 35-year-old 
student explained. 

Rodriguez, who attended 
classes at the adult school to 



Thursday, June 23, 1994 



learn English and become a U.S. 
citizen, accomplished two feats 
in one year; getting his high 
school diploma and being named 
Citizen of the Year by the State 
Consortium for Adult Education. 

Rodriguez came to the United 
States 14 years ago from 
Guatemala. At that time, he felt 
no immediate need to learn 
English since the people he 
spent time with spoke primarily 
Spanish and he listened to 
Spanish radio and TW. 

Later, he realized he needed to 
learn English in order to get his 
high school diploma. So, in 
1988 he enrolled in City Adult 
School where he proved to be a 
very diligent and focused 
student. He found time to study 
and participate in community 
activities while working two 
jobs and raising a family. 

According to Consortium 
president Gerry Delaney, “The 
adult school is full of students 
like Rodriguez who have many 
responsibilities, but Rodriguez 



stands out because he always 
gets the job done with a smile and 
in a timely fashion.” The 
Consortium acknowledged 
Rodriguez’ accomplishments by 
giving him the Citizen of the Year 
award. 

‘‘I feel proud. I wanted to do 
this as an example for my kids,” 
said Rodriguez, who has another 
daughter, age 7, and two sons, 15 
and 10. 

In fact, Rodriguez has set a 
good example not only at school 
but at work. Last year he was 
named Most Outstanding 
Employee at County Hospital 
where he works as a nurses’ aide. 

Receiving his diploma is only 
the first step to other ambitions 
for Rodriguez, who plans to 
continue his work in the medical 
field. “Now I want to go to 
college so I can become a nurse.” 

City News congratulates Oscar 
Rodriguez and wishes him 
continued success. 



adapted from Culver City News, June 23. 1994 



According to the article, why did Rodriguez get his high school diploma? 
Check (V) the three correct answers. 

to get a job 

to be able to go to college* 

to make his parents proud 

to graduate before his daughter* 

to set an example for his children* 



Prototype item notes 

Options B, D, and E, the keys, are reasons stated directly in the text for 
Rodriguez obtaining his diploma. Option A is contradicted by the text 
(Rodriguez already has a job) and Option C is unsupported by the text 
(his parents are not mentioned). 

Note: Prototype DM3 was modified based on pilot testing results and 
thus the estimated range and discrimination point are not available. 
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Reading item specifications 



Draw meaning 



f-r-\ 

DM Notes 



1. Item type DM1 could be constructed in multiple-choice format if there 
were enough pieces of specific information in the display to provide 
plausible distractors; that is, information of the same category (e.g., 
names of places in Prototype DM1 above) as the key. 

2. Item types such as DM2 and DM3 that require a response of a clause or 
more are not recommended in an open-ended format. Results from 
pretesting showed that open-ended responses were difficult to score 
because of the variety of answers provided. Moreover, many students 
responded to such questions by cop 3 ring large sections of the text, which 
provided little information about the students’ comprehension. 
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Reading item specifications 



Extract and combine information 



Bjxiract and Combine Information (EC) 



General description 

The test taker will extract and combine information from two or more 
propositions in one or more texts to construct meaning not directly 
stated in any one text. This skill is used, for example, in interpreting a 
problem stated in one text and locating its solution in another, or in 
combining instances of a phenomenon from two different sections of a 
text. 

Text type 

Display, paragraph, or passage 
Target level 

Intermediate low through advanced high 

Test level 

Level A: essential 
Level B: essential 

Item types 

In item type ECl, the test taker writes down a short answer to a question 
that requires combining information from two or more propositions in a 
primary text. In item type EC2, the test taker writes down a short 
answer to a question about a problem or condition described in an item 
stem using information from a primary text. In item type ECS, the test 
taker interprets information about a problem or condition described in 
an item stem and selects the referent that satisfies the problem or 
condition. In item type EC4, the test taker interprets two or more 
propositions from a secondary text associated with a particular referent 
found in a primary text and selects the correct referent from several 
options. 



Reading item specifications 



Extract and combine information 



Item Type ECl 



Item format 

Unique answer 

Stimulus attributes 

The primary text is a display, paragraph, or passage. 

The item stem is a factual question asking about information in the text 
that cannot be found within a single sentence but must be combined 
from two different places in the text to make a single meaning. For 
example, the question may ask for a number that represents a quantity 
of something mentioned in the text in different places. In this case, care 
must be taken that the information is presented in the text in such a way 
that it cannot easily be obtained by glancing through the text and 
combining numbers; instead, meaning must be drawn from different 
propositions in order to find the correct quantity. Mathematical ability 
should not be required to find the answer. 

The relevant part of the primary text contains target-level language. 
The item stem contains language at a lower level. 

Response attributes 

The test taker must write down the correct answer, which is a word or 
phrase not found in the text. The test taker is not penalized for 
misspellings or grammatical errors as long as the meaning is clear. 



Reading item specifications 



Extract and combine information 



Prototype item 

Read the story and answer the question. 



Strange True Stories 

Police arrested James Haskell, 
seventeen, in March after he tried to 
buy beer at Wally’s Mini-Mart by 
using a stolen driver’s license — that 
of Douglas Cleaver. The man 
behind the counter at Wally’s knew 
it was stolen because he is Douglas 
Cleaver, who had had his license 
taken in a truck break-in two 
months earlier. Haskell fled when 
he heard Cleaver talking to the 
police but left his wallet, which 
contained his real driver’s license. 



adapted from The Los Angeles Reader, June 2, 1994 



How many licenses did James Haskell have? 



Prototype item notes 

In order to find the correct answer (two), the test taker must interpret 
and combine information from two separate propositions in the text. 
Example responses appear below. 

Acceptable: 2; two; He has two licenses; He have two license (i.e., 

grammatical errors but meaning is clear) 

Unacceptable: 1; two months; his real driver’s license 
Estimated range: BL-AH Discrimination point: BH/IL 
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Reading item specifications 



Extract and combine information 



Item Type EC2 



Item format 

Unique answer 

Stimulus attributes 

The primary text is a display, paragraph, or passage. 

The item stem is a secondary text which consists of a statement written 
in the second person describing a scenario, followed by a factual 
question about the primary text. The statement sets up a condition or a 
problem, and the question asks the test taker to locate and use 
information in the primary text to satisfy the condition or solve the 
problem. 

Target-level language is found in the relevant part of the primary text or 
the item stem, or both. 

Response attributes 

The test taker must write down the correct answer to the question, 
which is a word or phrase not found in the primary text. The answer 
must demonstrate understanding of the relevant propositions, but does 
not need to be a complete sentence. The test taker is not penalized for 
misspellings or grammatical errors as long as the meaning is clear. 
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Reading item specifications 



Extract and combine information 



Prototype item 

Read the bicycle shop advertisement below and answer the question. 



( 3 % 

3b 



BIKES GALORE! 



3b 

3b 



BIKES GALORE 
of Southdale 
9000 Western Ave. 
456-0915 



Expert repairs — overnight service, most brands 
Affordable rentals — hourly and daily rates 
Lowest prices in town 
All cycling accessories and parts 
Exercise equipment ^ 

Insure your bicycle. Ask about our anti-theft policy. 

- 2 WESTSIDE LOCATIONS - 

BIKES GALORE 
of North Harbor 



V/SA 



ascovw 



FREE PARKING IN REAR 



3900 Vemer Blvd. 
278-1685 



HOURS: M-F 10-7 SAT 9-5:30 



adapted from Donnelly Yellow Pages , Los Angeles, 1992 



You took your bicycle to Bikes Galore 
on Friday for repairs. On what day 
will it be ready? 



Prototype item notes 

The test taker must interpret both the problem stated in the item stem 
and the phrase “overnight repairs” in the advertisement to get the 
correct answer. Example responses appear below. 

Acceptable: Saturday, the next day (both answers demonstrate 

comprehension of “overnight”) 

Unacceptable: Overnight (does not demonstrate comprehension of 
the word) 

Estimated range: BH-IH Discrimination point: IL/IH 
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Reading item specifications 



Extract and combine information 




Item Type ECS 



Item format 

Multiple choice 

Stimulus attributes 

The primary text is a display containing at least three members of a 
referent category (e.g., names, places) that can be distinguished from 
each other through their association with other information in the text. 

The item stem is a secondary text consisting of a statement written in 
the second person describing a scenario followed by a factual question. 
The statement sets up a condition or a problem, and the question asks 
the test taker to locate a unique referent in the primary text that satisfies 
the condition or solves the problem. 

Target-level language is found in the relevant part of the primary text or 
the item stem, or both. 

Response attributes 

A set of three or four response options in the form of a word or phrase 
follows the stem. The key is the correct referent. Distractors consist of 
other members of the same category (i.e., other persons or places) found 
in the text. 
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Reading item specifications 



Extract and combine information 



Prototype item 

Read the bicycle shop advertisements below. 



Village Bike Shop 



WE SHIP 
WORLDWIDE 



World Class Bicycles 
Call: 257-24 53 



854 Ocean Blvd • Palm Village 






• Bicycles for the entire family 

• Free lifetime service plan with purchase of any bike 

• All makes & models expertly serviced 

• Quality selection of accessories & apparel 

• Bicycle & exerciser rentals available 

Free Parking Available 
★ OPEN 7 DAYS ★ 







BIKES GALORE! 

Expert repairs — overnight service, most brands 
Affordable rentals — hourly and daily rates 
Lowest prices in town 
All cycling accessories and parts 
Exercise equipment 

Ir^sure your bicycle. Ask about our anti-theft policy. 






- 2 WESTSIDE LOCATIONS- 



BIKES GALORE 
of South dale 
9000 Western Ave. 
456-0915 

FREE PARKING IN REAR 



BIKES GALORE 
of North Harbor 
3900 Vemer Blvd. 
278-1685 
HOURS: M-F10-7 SAT 9-6:30 



VISA 



both ads adapted from Donnelly Yellow Pages , Los Angeles, 1992 



You want to send your cousin in Spain a bicycle as a gift. Which shop 
will send the bicycle for you? Circle the correct answer. 



Bikes Galore ^Village Bike Shop Both 



Prototype item notes 

The test taker must interpret both the problem stated in the item stem 
and the sentence “We ship worldwide” in the advertisement to find the 
correct answer. 



Estimated range: BL-IH 



Discrimination point: BH/IL 



Reading item specifications 



Extract and combine information 



Item Type EC4 



Item format 

Multiple choice 

Stimulus attributes 

The primary text is a display containing at least three members of a 
referent category (e.g., names, places) that can be distinguished from 
each other through their association with other information in the text. 

The item stem consists of a secondary text containing a paragraph or 
display that includes several pieces of information that uniquely identify 
a referent found in the primary text (i.e., the display) using different 
words than are found in the display, and a question asking which 
referent is being described. Care must be taken to ensure that the 
description contains enough features to identify one referent only. 

Target-level language is found in the relevant part of the primary text or 
the item stem, or both. 

Response attributes 

A set of three or four response options in the form of a word or phrase 
follows the stem. The key is the correct referent. Distractors consist of 
the other members of the same category (i.e., other persons or places) in 
the display. 



Reading item specifications 



Extract and combine information 



Prototype item 



Apartment Guide 



Apt. 

Features 


Willow 

Court 


Village 

Place 


Hamill 

Towers 


Pierce 

Plaza 


rent 


$750 


$550 


$700 


$950 


bdrms. 


2 


1 


2 


2 


baths 


2 


1 


1 


2 


parking 




♦ 


♦ 


♦ 


security bldg. 


♦ 




♦ 


♦ 


pool 


♦ 


♦ 






laundry 


♦ 




♦ 


♦ 


pets ok 




♦ 


♦ 




unique feature(s) 


• on bus line 


• all new 
appliances 


• all utilities 
paid 


• A/C 

• cable ready 



adapted from The Southern California Apartment Magazine, April 20-May 3, 1994 



Read this newspaper ad: 



Roommate Wanted 



Seeking female to share^ 
large sunny 2-bedroom apt 
w/non-smoker. Own bath. 
Security, washer/dryer, pool. 

Call Maria 555-8935 



Which apartment does the ad best describe? Circle the correct answer. 
*Willow Court Village Place Hamill Towers Pierce Plaza 



Prototype item notes 

The test taker must interpret the propositions in the secondary text, 
particularly the phrase “own bath” as well as the information in the 
advertisement to get the correct answer. 

Estimated range: BH-AL Discrimination point: BH/IL 
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Reading item specifications 



Extract and combine information 



EC Notes 



1. Item types ECl and EC2 could be constructed in multiple-choice format 
with plausible distractors either invented or found in the text. However, 
pretesting results indicated that the unique-answer format was very 
easy to score and reduces the guessing factor inherent in multiple- 
choice items. 

2. Item type EC3 and EC4 could be constructed in unique-answer format; 
however, results from pretesting this item type indicated that the 
plausible answers are restricted to the category members (e.g., “Willow 
Court” and “Village Place” in Prototype EC4 above) and it is much faster 
for the test taker to select an answer rather than write it in a blank. 
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Reading item specifications 



Interpret relationships 



Interpret Relationships (IR) 



General description 

The test taker will identify relationships between propositions in a text; 
that is, how the propositions function together to communicate a 
message. To perform this skill, the test taker must ordinarily be able to 
understand both the literal meaning of the relevant propositions in the 
passage and the logical connections between them. The relationships 
described below are appropriate for placement. 

Cause/effect: Given a cause found in a text, the test taker will identify 
its effect(s); or given an effect, the test taker will identify its cause(s). 

Compare/contrast: The test taker will identify similarities or 

differences between two referents in a text. 

Generalization/example: Given a generalization stated or implied in a 
text, the test taker will identify example(s) from the text; or given 
examples stated in a text, the test taker will make a generalization. 

Main idea/supporting details: The test taker will determine the relative 
importance of propositions in a text by distinguishing between main 
ideas and supporting details. 

Sequence of events: The test taker will arrange the propositions of a 
narrative text in chronological order. 

Text type 

Paragraph or passage 
Target level 

Intermediate low through advanced high 
Test levels 

Level A: essential; at least one IR item 

Level B: essential; at least one item assessing main idea/supporting 
details; other IR item t 3 q)es are optional 



Item types 

In item type IRl, the test taker chooses the statement that has the 
correct relationship with information in the item stem. In item type 
IR2, the test taker chooses three statements that have the correct 
relationship with information in the item stem. In item type IR3, the 
test taker selects the best summary of a text. In item t 3 q)e IR4, the test 
taker numbers a series of events from a narrative in chronological 
order. 



Reading item specifications 



Interpret relationships 



Item Type IRl 



Item format 

Multiple choice 

Stimulus attributes 

The primary text is a paragraph or a passage containing at least one 
stated or implied instance of a relationship of cause/effect, 
compare/contrast, or generalization/example. 

The item stem consists of a factual question presenting information 
stated or implied in the text and asking the test taker to identify other 
information from the text having a specified relationship with the given 
information, as follows: 

(a) Cause/effect: The item stem presents a cause and asks for its effect, 
or presents an effect and asks for its cause. 

(b) Compare/contrast: The item stem presents two referents from the 
text and asks how they are similar or dissimilar. 

(c) Generalization/example: The item stem presents a generalization 
and asks for an example, or presents an example and asks for a 
generalization. 

The relevant part of the primary text contains target-level language. 
The item stem contains language at a lower level. 

Response attributes 

A set of four single-phrase or single-sentence response options follows 
each stem. The response options contain language at a lower level than 
the target level. The test taker must circle the letter of the correct option. 
The key is an accurate paraphrase of the requested information. 

The three distractors are grammatically accurate and may reiterate 
words and phrases found in the text. The distractors consist of 
information supported by the text (i.e., accurate information about the 
text) and plausibly relate to the question but are inappropriate in that 
they do not adequately represent the specified relationship. 
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Reading item specifications 



Interpret relationships 



Prototype item 



City News 




City News' Person of the 
Week is adult school student 
Oscar Rodriguez, whose dream 
came true last week when he 
received his high school 
diploma after six years of study 
at City Adult School. 

Rodriguez said his primary 
motivation for graduating was 
his 17-year-old daughter, who 
becomes a high school senior 
this fall. “I wanted to have my 
diploma first,” the 35-year-old 
student explained. 

Rodriguez, who attended 
classes at the adult school to 



learn English and become a U.S. 
citizen, accomplished two feats 
in one year: getting his high 
school diploma and being named 
Citizen of the Year by the State 
Consortium for Adult Education. 

Rodriguez came to the United 
States 14 years ago from 
Guatemala. At that time, he felt 
no immediate need to learn 
English since the people he 
spent time with spoke primarily 
Spanish and he listened to 
Spanish radio and TV. 

Later, he realized he needed to 
learn English in order to get his 
high school diploma. So, in 
1988 he enrolled in City Adult 
School where he proved to be a 
very diligent and focused 
student. He found time to study 
and participate in community 
activities while working two 
jobs and raising a family. 

According to Consortium 
president Gerry Delaney, “The 
adult school is full of students 
like Rodriguez who have many 
responsibilities, but Rodriguez 



Thursday, June 23, 1994 

stands out because he always 
gets the job done with a smile and 
in a timely fashion.” The 
Consortium acknowledged 
Rodriguez’ accomplishments by 
giving him the Citizen of the Year 
award. 

“I feel proud. I wanted to do 
this as an example for my kids,” 
said Rodriguez, who has another 
daughter, age 7, and two sons, 15 
and 10. 

In fact, Rodriguez has set a 
good example not only at school 
but at work. Last year he was 
named Most Outstanding 
Employee at County Hospital 
where he works as a nurses’ aide. 

Receiving his diploma is only 
the first step to other ambitions 
for Rodriguez, who plans to 
continue his work in the medical 
field. “Now I want to go to 
college so I can become a nurse.” 

City News congratulates Oscar 
Rodriguez and wishes him 
continued success. 



adapted from Culver City News, June 23. 1994 



Circle the best answer for the following question. 



Why was Rodriguez chosen as Citizen of the Year? 



a. He studied a lot and accomplished two feats in one year. 

b. He had two jobs and set a good example for his children. 

*c. He worked hard to get an education and had a positive attitude, 
d. He attended City Adult School for six years and got his diploma. 



Prototype item notes 

Prototype IRl assesses a cause/effect relationship. All of the response 
options are accurate statements about the text that plausibly relate to the 
question. However, only the key (Option C) identifies the reason why 
Rodriguez was chosen as Citizen of the Year. 



Estimated range: IH-AH 



Discrimination point: AH+ 



Reading item specifications 



Interpret relationships 



Item Type IR2 



Item format 

Select three correct 

Stimulus attributes 

The primary text is a paragraph or a passage containing at least one 
stated or implied instance of the following: (a) a cause with a series of 

effects or vice versa; (b) two referents with a series of similarities or 
differences; or (c) a generalization with a series of supporting examples. 

The item stem consists of a factual question presenting information 
stated or implied in the text and asking the test taker to identify three 
statements about the text having a specified relationship with the given 
information, as follows: 

(a) Cause/effect: The item stem presents a cause and asks for three 

effects, or presents an effect and asks for three causes. 

(b) Compare/contrast: The item stem presents two referents from the 
text and asks for three similarities or differences. 

(c) Generalization/example: The item stem presents a generalization 
and asks for three examples that support the generalization. 

The relevant part of the primary text contains target-level language. 
The item stem contains language at a lower level. 

Response attributes 

A set of five single-phrase or single-sentence response options follows 
each stem. The response options contain language at a lower level than 
the target level. Each option is preceded by a blank. The test taker must 
check the three correct options. The keys are accurate paraphrases of 
the examples found in the text that support the generalization. 

The two distractors are grammatically accurate and may reiterate 
words and phrases found in the text. The distractors consist of 
information supported by the text (i.e., accurate information about the 
text) and plausibly relate to the question but are inappropriate in that 
they do not adequately represent the specified relationship. 



Reading item specifications 



Interpret relationships 



Prototype item 



City News 




City News' Person of the 
Week is adult school student 
Oscar Rodriguez, whose dream 
came true last week when he 
received his high school 
diploma after six years of study 
at City Adult School. 

Rodriguez said his primary 
motivation for graduating was 
his 17-year-old daughter, who 
becomes a high school senior 
this fall. “1 wanted to have my 
diploma first,” the 35-year-old 
student explained. 

Rodriguez, who attended 
classes at the adult school to 



learn English and become a U.S. 
citizen, accomplished two feats 
in one year: getting his high 
school diploma and being named 
Citizen of the Year by the State 
Consortium for Adult Education. 

Rodriguez came to the United 
States 14 years ago from 
Guatemala. At that time, he felt 
no immediate need to learn 
English since the people he 
spent time with spoke primarily 
Spanish and he listened to 
Spanish radio and TV. 

Later, he realized he needed to 
learn English in order to get his 
high school diploma. So, in 
1988 he enrolled in City Adult 
School where he proved to be a 
very diligent and focused 
student. He found time to study 
and participate in community 
activities while working two 
jobs and raising a family. 

According to Consortium 
president Gerry Delaney, “The 
adult school is full of students 
like Rodriguez who have many 
responsibilities, but Rodriguez 



Thursday, June 23, 1994 

stands out because he always 
gets the job done with a smile and 
in a timely fashion.” The 
Consortium acknowledged 
Rodriguez* accomplishments by 
giving him the Citizen of the Year 
award. 

“I feel proud. I wanted to do 
this as an example for my kids,” 
said Rodriguez, who has another 
daughter, age 7, and two sons, 15 
and 10. 

In fact, Rodriguez has set a 
good example not only at school 
but at work. Last year he was 
named Most Outstanding 
Employee at County Hospital 
where he works as a nurses* aide. 

Receiving his diploma is only 
the first step to other ambitions 
for Rodriguez, who plans to 
continue his work in the medical 
field. “Now I want to go to 
college so I can become a nurse.** 

City News congratulates Oscar 
Rodriguez and wishes him 
continued success. 



adapted from Culver dry News, June 23. 1994 



According to the article, what made it difficult for Rodriguez to get his 
high school diploma? Check (V) the three correct answers. 



He held two jobs.* 

He needed to learn English.* 

He was used to responsibility. 

He had to take care of his family.* 
He was named Citizen of the Year. 



Prototype item notes 

Prototype IR2 assesses a generalization/example relationship based on 
an implied generalization. All five options are accurate statements 
about the text, but only Options A, B, and D, the keys, support the 
generalization that it was difficult for Rodriguez to get his diploma. 

Note; Prototype IR2 was modified based on pilot testing results and thus 
the estimated range and discrimination point are not available. 
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Reading item specifications 



Interpret relationships 



Item Type IRS 



Item format 

Multiple choice 

Stimulus attributes 

The primary text is a paragraph or passage that contains a central idea 
or ideas, and several minor points or details. 

The item stem consists of the following question, with alternative 
wordings in brackets: 

Which paragraph [sentence] retells the most important idea[s] in the 
article [passage/story]? 

Target-level language is found in the primary text. 

Response attributes 

A set of four response options consisting of one to three sentences follows 
each stem. The response options contain language at a lower level than 
the target level. The test taker must circle the letter of the correct option. 
The key is an accurate paraphrase of the most important points 
discussed in the text. 

The three distractors are grammatically accurate and may reiterate 
words and phrases found in the text. The distractors consist of 
information supported by the text (i.e., accurate information about the 
text) but are inappropriate in that they are either too narrow in scope 
(i.e., they contain supporting details from the text rather than the most 
important ideas) or too broad in scope (i.e., they are more general than 
necessary to account for the most important ideas). 





Reading item specifications 



Interpret relationships 



Prototype item 



City News 



Thursday, June 23, 1994 




City News' Person of the 
Week is adult school student 
Oscar Rodriguez, whose dream 
came true last week when he 
received his high school 
diploma after six years of study 
at City Adult School. 

Rodriguez said his primary 
motivation for graduating was 
his 17-year-old daughter, who 
becomes a high school senior 
this fall. “I wanted to have my 
diploma first,” the 35-year-old 
student explained. 

Rodriguez, who attended 
classes at the adult school to 
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learn English and become a U.S. 
citizen, accomplished two feats 
in one year: getting his high 
school diploma and being named 
Citizen of the Year by the State 
Consortium for Adult Education. 

Rodriguez came to the United 
States 14 years ago from 
Guatemala. At that time, he felt 
no immediate need to learn 
English since the people he 
spent time with spoke primarily 
Spanish and he listened to 
Spanish radio and TV. 

Later, he realized he needed to 
learn English in order to get his 
high school diploma. So, in 
1988 he enrolled in City Adult 
School where he proved to be a 
very diligent and focused 
student. He found time to study 
and participate in community 
activities while working two 
jobs and raising a family. 

According to Consortium 
president Gerry Delaney, “The 
adult school is full of students 
like Rodriguez who have many 
responsibilities, but Rodriguez 

e following question. 



stands out because he always 
gets the job done with a smile and 
in a timely fashion.” The 
Consortium acknowledged 
Rodriguez’ accomplishments by 
giving him the Citizen of the Year 
award. 

“I feel proud. I wanted to do 
this as an example for my kids,” 
said Rodriguez, who has another 
daughter, age 7, and two sons, 15 
and 10. 

In fact, Rodriguez has set a 
good example not only at school 
but at work. Last year he was 
named Most Outstanding 
Employee at County Hospital 
where he works as a nurses’ aide. 

Receiving his diploma is only 
the first step to other ambitions 
for Rodriguez, who plans to 
continue his work in the medical 
field. “Now I want to go to 
college so I can become a nurse.” 

City News congratulates Oscar 
Rodriguez and wishes him 
continued success. 



from Culver City News. June 23, 1994 



Which paragraph retells the most important ideas in the article? 

a . When Rodriguez came to the United States he didn’t speak English and most of his 
friends only spoke Spanish. After many years of study, he has learned to speak 
English and has earned a high school diploma. 

*b. Rodriguez received his high school diploma and was named Citizen of the Year 
because he overcame many obstacles to make his dream come true. He hopes his 
accomplishments will serve as an example to his children. 

c. Rodriguez works two jobs to support his family. Because of his excellence at County 
Hospital where he works as a nurse’s aide, he was named Most Outstanding 
Employee. Rodriguez has set a good example for children. 

d. Rodriguez studied at City Adult School for six years. He was motivated to get his 
high school diploma by his oldest daughter who will be a high school senior this 
year. He felt it was important to graduate before she did. 



Prototype item notes 

Each option contains factual information about the text; however, only 
the key (Option B) contains the main idea. Options A, C, and D are too 
narrow in scope. 

Estimated range: IL-AH 



Discrimination point: IL/IH 
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Item format 
Sequencing 

Stimulus attributes 

The primary text is a narrative passage which has at least five discrete 
events. The events must not be presented in chronological order within 
the passage. 

The item stem begins with the following instructions, with words in 
brackets indicating that modifications can be made here as appropriate 
to the text: 

Below are some important events from [the story]. Number the 
events (1, 2, 3, 4, 5) in the order they happened. The first one is done 
for you. 

A set of five sentences follows the instructions. Each sentence describes 
one of the events mentioned in the text. The events are presented in 
random order. 

The primary text contains target-level language. The sentences in the 
item stem contain language at a lower level. 

Response attributes 

Each sentence is preceded by a blank, in which the test taker must write 
a number corresponding to the chronological ordering of the events in 
the narrative. The first event contains a 1 in the blank. 
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Prototype item 

Read the story and answer the question. 



Strange True Stories 

Police arrested James Haskell, 
seventeen, in March after he tried to 
buy beer at Wally’s Mini-Mart by 
using a stolen driver’s license — that 
of Douglas Cleaver. The man 
behind the counter at Wally’s knew 
it was stolen because he is Douglas 
Cleaver, who had had his license 
taken in a truck break-in two 
months earlier. Haskell fled when 
he heard Cleaver talking to the 
police but left his wallet, which 
contained his real driver’s license. 



adapted from The Los Angeles Reader^ June 2, 1994 



Below are some important events from the story. Number the events 
(1, 2, 3, 4, 5) in the order they happened. The first one is done for you. 



key 

2 

3 

1 _1 
5 

4 



James Haskell tried to buy some beer at Wally’s Mini-Mart. 
Douglas Cleaver called the police. 

Someone broke into Douglas Cleaver’s truck. 

The police found James Haskell and arrested him. 

James Haskell ran away from Wally’s. 



Prototype item notes 

The test taker must reconstruct the chronological order of the events in 
the story, which are not narrated in the order in which they occurred. 

Estimated range: IL-AH Discrimination point: IL/IH 
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IR Notes 



1. IR item types that require the test taker to explain complex relationships 
are not recommended in an open-ended format. Results from pretesting 
showed that when asked to respond in an open-ended format, test takers 
either copied large sections of the text or provided no response at all. 
When responses were provided they were difficult to score because of the 
variety of responses given. 

2. Item type IR4 was pretested with and without the first event indicated. 
Pretesting results showed that indicating the first event seemed to help 
orient test takers to the task and did not limit the ability of the item to 
discriminate at the upper levels. This item type was also pretested and 
pilot tested with a multi-paragraph passage using six events rather 
than five. Pilot testing results suggested that ordering six events makes 
the item too difficult for the population. 
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Make Inferences (MI) 



General description 

The test taker will draw conclusions about information implied but not 
explicitly stated in a text. 

Text type 

Paragraph, multi-paragraph passage 
Target level 

Intermediate high to advanced high 

Test level 

Level A: optional 
Level B: essential 

Item types 

MI item tjrpes were not developed as part of the prototyping effort. Texts 
that lend themselves to assessing this skill need to be selected and item 
types developed following the prototyping process described in Kahn et 
al. (1995). 
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Overview 



Writing T(isk Specifications 
Overview 

This section provides specifications for developing writing tasks that 
require the test taker to generate (GN) a writing sample of at least one 
paragraph in length. The section is organized as follows. First, an 
introductory page provides the following information: 

General description 

A brief definition of the skill as operationalized in task types and a 
description of task elements 

Test level characteristics 

A discussion of the relationship between test level (A, B or both) and 
elements of a writing task 

Tcwk types 

A brief description of each task type 




The introductory page is followed by specifications for each task type. The 
specifications include the following information; 

Test level 

The appropriate test level (A, B or both) for the task 
Stimulus attributes 

The characteristics of the input to the test taker, which include: 

(a) a description of the task 

(b) specific information about the primary text and, where appropriate, 
the secondary text 

(c) the language characteristics of the prompt and, where appropriate, 
the secondary text 

(d) a description of the space provided for the response 
Response attributes 

The characteristics of the expected response, including the content and 
linguistic features of a model response 
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Prototype task 

A task that exemplifies the task t5T)e 

All prototype tasks were pilot tested in the form in which they are 
presented and were found to be promising as models for placement test 
development based on statistical performance and content 
considerations. 

Note: The facsimiles showing the space provided for the test taker’s 
response have been reduced in this document. On an operational test, 
facsimiles should be larger. 



Prototype task notes 

Explanatory notes about the prototype task, detailing the audience and 
purpose of the task, the source of informational content, and, where 
appropriate, how that information needs to be transformed in the 
response 



Model response 

An actual response to the prototype task from pilot testing representing 
the highest score on the scoring rubric 

Note: See Scoring GN Tasks for scoring procedures and rubrics. 



Following the writing task specifications, additional notes about the task 
types are provided. Recommended procedures for scoring GN tasks and 
scoring rubrics for Level A and Level B are presented at the end of the 
section. 
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Generate (GN) 



General description 

The test taker will generate a written response of a paragraph or more 
in length to a prompt of two to four sentences that explicitly specifies the 
elements of the writing task including audience, purpose, form, and 
source of informational content. These elements are defined below. 

Audience 

The addressee, which can be personal (e.g., friend, family, pen pal) 
or institutional (e.g., landlord, newspaper, business, potential 
employer) 

Purpose 

The communicative goal of the task 
Form 

The genre (e.g., note or letter) of the expected response 
Informational content 

The primary source of the information to be presented in the 
response, which can be one of the following: (a) the prompt itself; 
(b) the primary text; (c) a secondary text; or (d) personal experience. 
Whatever the source, it is essential that the writing task be 
constructed in such a way that the test taker cannot simply copy 
information from the source to adequately fulfill the task. 

The prompt also provides a context for the writing task by relating the 
task to a previously read primary text, either by providing some or all of 
the informational content for the writing task or by serving as a 
springboard to set up a related problem or scenario. 

The task is authentic and communicative in that it represents tasks that 
adult ESL students might want or need to carry out outside of the ESL 
classroom environment. It is cultiirally accessible to the majority of test 
takers and elicits writing samples that can be scored quickly and 
reliably. 

Writing samples are judged holistically on task fulfillment and use of 
English. 
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Test level characteristics 

A given writing task can be used to assess a range of levels since it is 
scored on a scale tied to specific proficiency levels. A task is targeted at 

a-range-of-levels-so-t-hat-test-takers-at-the“lowest“levekwithin“thre“range’ 

are able to provide a scoreable response even if they cannot adequately 
fulfill the task, and test takers at the highest level can fulfill the task 
successfully both in terms of content development and linguistic control 
appropriate for that level. 

Tasks that only appear on Level A tests are targeted from beginning 
high through intermediate high.^i Tasks that only appear on Level B 
tests are targeted from intermediate low through advanced high. Tasks 
that could appear on both Level A and Level B tests are targeted from 
beginning high through advanced high. 

The targeted level of a writing task is determined by a number of factors, 
as specified in Figure 2 below. 

Figure 2 

Characteristics of Level A and Level B Writing Tasks 



Level 


Length 

(Form) 


Audience 


Primary 

Function 


Primary 

Text 


Informational 

Content 


A 


Paragraph 
(e.g., note) 


Personal 


Social 

Factual^ 


Display^ 

Paragraph 


Prompt 

Personal Experience^ 


A&B 


Passage 
(e.g., letter) 


Personal 


SociaP 

Factual 


Paragraph 


Primary Text 
Personal Experience^ 
Secondary Text^ 


B 


Passage 
(e.g., letter) 


Institutional 


Factual^ 

Suasive 


Passage 


Personal Experience 
Secondary Text 



^ has not been developed 



Task types 

In task type GNl (Level A), the test taker writes a short note fulfilling a 
social function. In task type GN2 (Levels A and B), the test taker writes 
a personal letter using information presented in a primary text to fulfill 
a factual function. In task type GN3 (Level B), the test taker writes an 
institutional letter drawing on personal experience to fulfill a suasive 
function. In task type GN4 (Level B), the test taker writes an 
institutional letter using information presented in a secondary text to 
fulfill a suasive function. 



^ ^Beginning low students may be able to copy limited written materials or to complete a 
simple form but are not expected to be able to generate a writing sample. 
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Generate 



) 



Task Type GNl 



Test level 

Level A 

Stimulus attributes 

The task is to write either a personal or institutional note to fulfill a 
basic social function using specific information provided in the prompt. 

The primary text is a display or a paragraph appropriate for Level A test 
takers. 

The language of the writing prompt is simple and clear using 
structures and vocabulary appropriate for students entering beginning 
high. 

A facsimile of a ruled note pad containing six to ten lines is provided for 
the test taker’s response. 

Response attributes 

The test taker must generate an appropriate response to the writing task 
in the space provided. The responses are scored by trained raters using 
a clearly articulated scoring rubric. 

The model response is a series of related sentences that form one or 
more paragraphs and fulfill all the task requirements. The model 
response demonstrates adequate control over simple structures, 
although there may be some errors. Complex structures may be 
attempted. The model response may include a few words or phrases 
directly copied from the primary text or the prompt but these are usually 
incorporated into the test taker’s own language. 
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Prototype task 









The Department ot Parks and Recreation 
is sponsoring free Saturday evening concerts 

July 9, 16 and 23. 

Concerts will begin at 7 p.m. and will last about two hours. 
They will be held in the new outdoor amphitheater. 
Call 555-8632 for more information. 



adapted from Elk Grove Adult School ESL Placement Test 



Your friend saw the announcement above and asked you to go to the 
concert on July 16th. You can’t go that evening. Write a note 
explaining the problem and ask your fHend to go with you on the 23rd. 
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Prototype task notes 

The audience is personal (a friend) and the purpose is to explain a 
problem and make a suggestion. 

The test taker does not need to use the information from the text per se 
in the response, although he or she may use language from the prompt. 
If language from the prompt is used, it must be transformed from the 
second person to the first person. 

Model response 



Just a note . . . 



V\f^ o->rv<»y\A ■ X 



\r,A- X Qjxme . 



JQ^ 









VVvp. siiL. Aula 16 -hk- 

'H Vri sn Ld. A. Jr. 



f- 



tF luJI he u(S(^ \rrri\l\ 

)iorJSC, Jyy/ r/Zr?/?/ /iq ^ Me 







,^/ho-f r/a ^ ad y^/!j 



tJr//^ 23 'yJ ? p'^crSG 



Wf/o 



Una a/ if uygr/?'/ u///j /7?^ 
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Task Type GN2 



Test level 

Level A and Level B 
Stimulus attributes 

The task is to write a personal letter to fulfill a factual function by 
retelling specific information presented in the primary text in the first 
person. 

The primary text is a paragraph targeted at the intermediate levels and 
appropriate for both Level A and Level B test takers. The paragraph is 
narrated in the third person and contains at least three discrete events 
that are not presented in chronological order. 

The language of the writing prompt is simple and clear using 
structures and vocabulary appropriate for students entering beginning 
high. 

A facsimile of a ruled sheet of paper containing fifteen to twenty lines is 
provided for the test taker’s response and includes a salutation and 
opening sentence to orient the test taker to the task. 

Response attributes 

The test taker must generate an appropriate response to the writing task 
in the space provided. The responses are scored by trained raters using 
a clearly articulated scoring rubric. 

The model response for Level A is a series of related sentences that form 
one or more paragraphs and fulfill all the task requirements. The 
model response demonstrates adequate control over simple structures, 
although there may be some errors. Complex structures may be 
attempted. The model response may include a few words or phrases 
directly copied from the primary text or the prompt but these are usually 
incorporated into the test taker’s own language. 

'The model response for Level B is a series of related sentences that form 
one or more paragraphs and fulfill all the task requirements. Most 
ideas are effectively integrated and developed. The model response 
contains a variety of simple and complex structures with few or no 
errors in simple structures and possibly some errors with complex 
structures. Advanced vocabulary is used, although there may be some 
errors in form and usage. 
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Prototype task 



Strange True Stories 

Police arrested James Haskell, 
seventeen, in March after he tried to 
buy beer at Wally’s Mini-Mart by 
using a stolen driver’s license — that 
of Douglas Cleaver. The man 
behind the counter at Wally’s knew 
it was stolen because he is Douglas 
Cleaver, who had had his license 
taken in a truck break-in two 
months earlier. Haskell fled when 
he heard Cleaver talking to the 
police but left his wallet, which 
contained his real driver’s license. 

adapted from The Los Angeles Reader, June 2, 1994 

Imagine you are Douglas Cleaver. Write a letter to a friend explaining 
the strange thing that happened to you at Wally’s Mini-Mart. The letter 
has been started for you. 



Dear Chris, 


You won’t believe what happened to me the other day! 




























’ 
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Prototype task notes 

The audience is personal (a friend) and the purpose is to communicate 
details of a strange event. 

The test taker needs to use the information from the text in the response, 
although it must be transformed in person (third to first), in register 
(from journalistic to informal) and in ordering the events, which also 
requires transforming the syntax. 

Model response: Level A 



Dear Chris, 


You won’t believe what haooened to me the other day! 


J! VdoSJUia mW.' 0^^ . 


i 


ry/X jbuCJt/AAJL . 


J2>LCtYV>ax f\XDX- • 


J. CM)^ It- (sxjxJL \ 




S CAjditA/ tiASL. fL&tuJL' , 


Jl iJ> tJUS- /LOAV . 


xHaJL 0/U^ OA/UA^jUiC 


fU^UcS. . ^jl, COAijiJi' ijOUAy . 
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Model response: Level B 



Dear Chris, 



You won’t believe what happened to me the other dayl Ai 






h/iHi rvwj driurs licenle 



T alMosi -for^offe^ nhotrt 



HuJ- hoi/v 



^qrbnrecf TcaiV ^jfclnverj 

. A I A ^ ^ ^ J. ^ 



/iceU-e Ci^(Kfv^!0/ie 



lu^ itif if) 'H\t Afitti-AfArf i^htr^ I 
■ //e drivers. Itccn^^ 



£ind Tc,aMJ rrwj ovjy^ hCLme //. ^ 



Calkd ikB f:^ol!ct cffcftice^ hui 'fhai' 



^hB h 

I'fH-lt ihitr -f-Udl ut/he4i Ae /tecire^ 



H> 4hB police , I />e !^^i£ 



\/erv -fr/ohjifitd ^ hBtcu422 hi, e^en 



very rnot 
^or^oi ivli 



fil£ £>iv*i ivOtllt^ ih C( 



h vfr\j . There h li driv^rk he ett&e 



m HfOi't vyOifM. cfit4rse^^e 

A/re^ied yen/ (j-oon. 



Z/ ^orithodv Aad 4old me he^fore. 



iha-h ^uth even-h ton Id tdriH 

Z i^ouldh / ha\^Q bdft ve cl oecau^eldort '^ktt>eyt 
co’incidtnce^ bui i f did Acippe^. I fc tvrl'h/oi4 atPui-iH 

WMdo yo\4 ’Hiihk ahouf i4r Id 

*v«/f 4or your U'Htr. /DOd^/As, 
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Task Type GN3 



Test level 
Level B 

Stimulus attributes 

The task is to write an institutional letter that fulfills a suasive function 
drawing upon personal experience. 

The primary text is a paragraph or multi-paragraph passage 
appropriate for Level B test takers. 

The language of the writing prompt is simple and clear, using 
structures and vocabulary appropriate for students entering 
intermediate low. 

A facsimile of a ruled sheet of paper containing fifteen to twenty lines is 
provided for the test taker’s response and includes a salutation and 
opening sentence to orient the test taker to the task. 

Response attributes 

The test taker must generate an appropriate response to the writing task 
in the space provided. The responses are scored by trained raters using 
a clearly articulated scoring rubric. 

The model response is a series of related sentences that form one or 
more paragraphs and fulfill all the task requirements. Most ideas are 
effectively integrated and developed. The model response contains a 
variety of simple and complex structures with few or no errors in simple 
structures and possibly some errors with complex structures. 
Advanced vocabulary is used, although there may be some errors in 
form and usage. 
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Prototype task 



City News 




City News" Person of the 
Week is adult school student 
Oscar Rodriguez, whose dream 
came true last week when he 
received his high school 
diploma after six years of study 
at City Adult School. 

Rodriguez said his primary 
motivation for graduating was 
his I7-year-oId daughter, who 
becomes a high school senior 
this fall. “I wanted to have my 
diploma first,” the 35-year-old 
student explained. 

Rodriguez, who attended 
classes at the adult school to 



Thursday, June 23, 1994 



learn English and become a U.S. 
citizen, accomplished two feats 
in one year: getting his high 
school diploma and being named 
Citizen of the Year by the State 
Consortium for Adult Education. 

Rodriguez came to the United 
States 1 4 years ago from 
Guatemala. At that time, he felt 
no immediate need to learn 
English since the people he 
spent time with spoke primarily 
Spanish and he listened to 
Spanish radio and TV. 

Later, he realized he needed to 
learn English in order to get his 
high school diploma. So, in 
1988 he enrolled in City Adult 
School where he proved to be a 
very diligent and focused 
student. He found time to study 
and participate in community 
activities while working two 
jobs and raising a family. 

According to Consortium 
president Gerry Delaney, “The 
adult school is full of students 
like Rodriguez who have many 
responsibilities, but Rodriguez 



stands out because he always 
gets the job done with a smile and 
in a timely fashion.” The 
Consortium acknowledged 
Rodriguez’ accomplishments by 
giving him the Citizen of the Year 
award. 

“I feel proud. I wanted to do 
this as an example for my kids,” 
said Rodriguez, who has another 
daughter, age 7, and two sons, 15 
and 10. 

In fact, Rodriguez has set a 
good example not only at school 
but at work. Last year he was 
named Most Outstanding 
Employee at County Hospital 
where he works as a nurses’ aide. 

Receiving his diploma is only 
the first step to other ambitions 
for Rodriguez, who plans to 
continue his work in the medical 
field. “Now I want to go to 
college so I can become a nurse.” 

City News congratulates Oscar 
Rodriguez and wishes him 
continued success. 



from Culver City News, June 23, 1994 
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Who would you choose to be Person of the Week? Write a letter to City 
News about a friend, family member, or someone in the news who you 
think should be the next Person of the Week. Explain your reasons for 
recommending this person. The letter has been started for you. 



2>eo4- QU4f^ editor: 
9 uu>uld IJie tii’ 



ad.UtA 



ttesU PeMwu (^|f Uie We&k. 
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Prototype task notes 

The audience is institutional (a newspaper) and the purpose is to 
persuade the newspaper to select the proposed nominee. 

The test taker must use personal experience to develop a persuasive 
argument. Information from the text is not needed per se, although 
some of the ideas, vocabulary, and syntax may be transformed and used. 

Model response 



^bean> /^hiu4> SdUo^: 

9 iWudctUk&ti^neoomme*ui f^r. 






*iAxi Pendoti ojf tU& Week. 






lPf)i>yvX" t/o Ok. ^ 






D — -T 

jXtLJX ^ 

\Ow*<-jL c4U?cfr-e.»i 









tv 9 



Jo 



"h> Oi/T<. 

gO >^»v» 

urC.<« ^ u>v^ 

tC».y ^TW^.lC*\ A^J l>/w 

•£^- 






* .J~ 



^i^fU 7 ^ 

<XLyjn . !ld 

Wy^^/4/tv C>yw^a.«.X*A aJcJQi , l«*W» a 

UvAt L*yrJi » Ay<> Eri*^ v^^^ 

i2i/Ki So ^ ,^*~ 

f^h ^ y\x^T~ ^ { /JjjeJ t. . 

ik 
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Task Type GN4 



J 



Test level 
Level B 

Stimulus attributes 

The task is to write an institutional letter that fulfills a suasive function 
using specific information presented in a secondary text. 

The primary text is a paragraph or multi-paragraph passage 
appropriate for Level B test takers. 

The secondary text is a display such as a table or a list containing four to 
six pieces of specific information about the situation introduced in the 
prompt. 

The language of the prompt and secondary text is simple and clear, 
using structures and vocabulary appropriate for students entering 
intermediate low. Information in the secondary text is presented in 
words and short phrases, not complete sentences. 

A facsimile of a ruled sheet of paper containing fifteen to twenty lines is 
provided for the test taker’s response and includes a salutation and 
opening sentence to orient the test taker to the task. 

Response attributes 

The test taker must generate an appropriate response to the writing task 
in the space provided. The responses are scored by trained raters using 
a clearly articulated scoring rubric. 

The model response is a series of related sentences that form one or 
more paragraphs and fulfill all the task requirements. Most ideas are 
effectively integrated and developed. The model response contains a 
variety of simple and complex structures with few or no errors in simple 
structures and possibly some errors with complex structures. 
Advanced vocabulary is used, although there may be some errors in 
form and usage. 








83 



Writing task specifications 



Generate 



Prototype task 
City News 



Thursday, June 23, 1994 




City News' Person of the 
Week is adult school student 
Oscar Rodriguez, whose dream 
came true last week when he 
received his high school 
diploma after six years of study 
at City Adult School. 

Rodriguez said his primary 
motivation for graduating was 
his 17-year-old daughter, who 
becomes a high school senior 
this fall. wanted to have my 
diploma first,” the 35-year-old 
student explained. 

Rodriguez, who attended 
classes at the adult school to 



learn English and become a U.S. 
citizen, accomplished two feats 
in one year: getting his high 
school diploma and being named 
Citizen of the Year by the State 
Consortium for Adult Education. 

Rodriguez came to the United 
States 14 years ago from 
Guatemala. At that time, he felt 
no immediate need to learn 
English since the people he 
spent time with spoke primarily 
Spanish and he listened to 
Spanish radio and TV. 

Later, he realized he needed to 
learn English in order to get his 
high school diploma. So, in 
1988 he enrolled in City Adult 
School where he proved to be a 
very diligent and focused 
student. He found time to study 
and participate in community 
activities while working two 
jobs and raising a family. 

According to Consortium 
president Gerry Delaney, ‘The 
adult school is full of students 
like Rodriguez who have many 
responsibilities, but Rodriguez 



stands out because he always 
gets the job done with a smile and 
in a timely fashion.” The 
Consortium acknowledged 
Rodriguez’ accomplishments by 
giving him the Citizen of the Year 
award. 

‘‘I feel proud. I wanted to do 
this as an example for my kids,” 
said Rodriguez, who has another 
daughter, age 7, and two sons, 15 
and 10. 

In fact, Rodriguez has set a 
good example not only at school 
but at work. Last year he was 
named Most Outstanding 
Employee at County Hospital 
where he works as a nurses’ aide. 

Receiving his diploma is only 
the first step to other ambitions 
for Rodriguez, who plans to 
continue his work in the medical 
field. ‘‘Now I want to go to 
college so I can become a nurse.” 

City News congratulates Oscar 
Rodriguez and wishes him 
continued success. 



from Culver City News, June 23. 1994 



Person of the Week: JaneJeunKim 



Year 


E}veiit 


1964 


Born in Korea 


1985 


Arrival in the United States 


1986-1988 


Medical assistant training program 


1989-present 


Medical assistant at City Family Clinic 


1991-present 


Korean-American Youth Services volunteer 


1993-present 


Instructor of community first-aid classes 
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Here is some information about Jane Jeun Kim. Use the information to 
write a letter to City News explaining why she should be the next Person 
of the Week. The letter has been started for you. 



CU^ Ollor: 

S ivouid to recommtnJ ^ana ^un~Kin\ ai ika naxi perion of tka 'KJaeL, 
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Prototype task notes 

The audience is institutional (a newspaper) and the purpose is to 
persuade the newspaper to select the given nominee. 

The test taker must use information provided in the secondary text in 
the responses but must also go beyond simply repeating the information 
to develop a persuasive argument. Some of the vocabulary needed to 
fulfill the task may be taken from the secondary text, but it must be 
incorporated into the test taker’s own language. Information from the 
primary text per se is not needed, although some of the vocabulary and 
syntax may be transformed and used. 

Model response 



:/r, 



^ wou UMs. to rocomnwrul ^ano ^un J(un cla tka fuixi j^r6on JiL IILL 

-tp 'tU 

oLec^tLtftJ "ti* OL. h*jtci^C4ii£ ^ -o-o 

isD 't'kc 

i%t*a ■^vt.cJce^-y'^CC^ 

Mupj ^ ^ ^ 

ari u^4e-r-^ <rLe cty»ytcs . 

eu<-^e>Hjt^ job «go clu i^%.-&rv4ct'O r' 



Ouj. 






An 



^ -firt.'t- oJd o&^'Xe^ usL&tjl U 

>CeeLn*^ -fro ftit, Lt^ Tu^ o tAe/Li'k^Atx, ^ 'i*Lje. o\ A «<« 

/fytt,cnicn.i\. .Se^v/Og^^ ujLa^-^ 

yy /^i/CUCX>hr%^ 

^-/c. su 



vsXe ^ <X4>\^ H^k0^^jct 'Pe^x 



Ot, of 'tU*. 
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) 



GN Notes 



1. A variant of task type GNl was pretested in which the test taker was 
instructed to tell a friend about the concerts and invite him to go on a 
specific day. This task was less successful because the test taker could 
simply copy much of the information from the primary text to fulfill the 
task. 

2. Writing tasks GN3 and GN4 are variants of the same task drawing on 
different sources of informational content. Pilot testing the examples 
showed advantages and disadvantages of each task type. (See Kahn, 
forthcoming, for a detailed discussion). 
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Scoring GN Tasks 



Overview 

This section presents recommended procedures for holistically scoring 
writing tasks that reqiiire the test taker to produce a sample of at least 
one paragraph in length. 

The section is organized as follows. First, recommended scoring 
procedures are discussed. Then, Level A and Level B scoring rubrics 
are provided including a description of rubric features and an 
explanation of how the rubrics function. 

Scoring procedures 

Each writing sample is scored independently by two trained raters. (See 
Appendix E for recommended training procedures.) 

Scores are recorded using numbers associated with Model Standards 
proficiency levels (i.e., 1 = BL, 2 = BH, 3 = IL, 4 = IH, 5 = AL, 6 = AH). If 
the two scores are within one point, the scores are averaged. If the two 
scores are more than one point apart, the sample should be read by a 
third trained rater to determine the appropriate score assignment. If a 
third rater is not available, the two raters should discuss the sample 
and come to agreement about the score assignment. 
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Level A Rubric 

The Level A rubric is designed to assess test-taker performance on writing 
tasks targeted from beginning low through intermediate high. Writing 
ability is assessed on two major dimensions: task fulfillment (columns) 
and language (rows). 

Task fulfillment refers to the degree to which the test taker successfully 
completes the requirements of the task. Decisions about task fulfillment 
fall into three categories: complete, partial, and not at all. These 

categories are defined below. 

Not at all: None of the task requirements are fulfilled 

Partial: 'Some requirements of the task are fulfilled, but others 

are either not fulfilled adequately or are not addressed 

Complete: All requirements of the task are fulfilled 

Language is evaluated based on the following characteristics: 

(a) amount of text generated by the test taker 

(b) amount of language copied from prompt or text 

(c) paragraph development 

(d) use of complex structures and advanced vocabulary 

(e) control over simple structures 

Raters determine the degree of task fulfillment and find the band of 
language descriptors that best fits the sample of writing. The cell at which 
decisions on these two dimensions intersect corresponds to a proficiency 
level. For example, a writing sample that fulfills some requirements of the 
task (partial column) and contains language consisting of simple sentences 
that form a loosely-organized paragraph (third row from the bottom) would 
be given a score of 3 (IL), as indicated in the cell where the column and row 
intersect. 

If the test taker fulfills all requirements of the task and the writing sample 
demonstrates good control over simple structures and contains some 
complex structures and advanced vocabulary, the test taker should be 
retested with Level B tasks to determine appropriate placement, as 
indicated in the cell where the complete column and the top row intersect. 
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Level A Rubric 



Task Fulfillment 

Language 


Not at all 


Partial 


Complete 


• variety of simple sentences; complex sentences 
usually attempted 

• advanced vocabulary and complex verb tenses; 
some errors in form and usage 

• few errors in simple structures; frequent errors in 
complex structures 

OR more proficient 






retest 
Level B 


• related sentences form a simple sequential 
paragraph 

• complex structures attempted; some errors still 
present in simple structures 

• a few words or phrases may be directly copied 
from prompt or text but they are usually 
incorporated into test taker^s own language 


3 

(IL) 


4 

(IH) 


4 

(IH) 


• simple related sentences form a short, loosely 
organized paragraph; errors may be frequent 

• some words, phrases, or sentences may be directly 
copied from prompt or text, but enough student 
generated language to evaluate 


2 

(BH) 


3 

(IL) 


3 

(IL) 


• one or two simple, student generated sentences; 
sentences may not be related; errors are frequent 

• several words, phrases, or sentences may be 
directly copied from prompt or text, but not enough 
student generated language to evaluate 


2 

(BH) 


2 

(BH) 




• less than a single sentence; words or phrases may 
not be related 

• sample may contain isolated words or phrases 
directly copied firom prompt or text 

OR no response 


1 

(BL) 







I 



Task Fulfillment Definitions 

Not at all: None of the task requirements are fulfilled 

Partial: Some requirements of the task are fulfilled, but others are either not fulfilled 

adequately or are not addressed 

Complete: All requirements of the task are fulfilled 
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Level B Rubric 



The Level B rubric is designed to assess test-taker performance on writing 
tasks targeted from intermediate low through advanced high. Writing 
ability is assessed on two major dimensions: task fulfillment/content 

development (colvunns) and language (rows). 

Task fulfillment refers to the degree to which the test taker successfully 
completes the requirements of the task. Content development refers to how 
well the test taker develops and integrates ideas. Decisions about task 
fulfillment/content development fall into four categories: complete, partial, 
minimal, and insufficient. These categories are defined below. 



Insufficient: Few, if any, of the task requirements are fulfilled; ideas 
are not integrated or developed 



Minimal: 

Partial: 

Complete: 



Some requirements of the task are fulfilled; ideas are 
loosely integrated but may lack development 

Most requirements of the task are fulfilled; ideas are 
usually integrated and developed 

All requirements of the task are fulfilled; most ideas are 
effectively integrated and developed 



Language is evaluated based on the following characteristics: 

(a) balance between simple and complex sentences 

(b) use of complex verb tenses and advanced vocabulary 

(c) control over simple and complex structures 

(d) amount of language copied from prompt or text 



Raters determine the degree of task fulfillment/content development and 
find the band of language descriptors that best fits the sample of writing. 
The cell at which decisions on these two dimensions intersect corresponds 
to a proficiency level. For example, a writing sample that fulfills some 
requirements of the task while loosely integrating ideas without much 
development (minimal column) and contains language consisting of 
predominantly simple sentences with some attempt at complex verb tenses 
(third row from the bottom) would be given a score of 4 (IH), as indicated in 
the cell where the col\imn and row intersect. 

If the test taker fulfills few requirements of the task and the writing sample 
contains predominantly copied text or just a few simple sentences, the test 
taker should be retested with Level A tasks to determine appropriate 
placement, as indicated in the cell where the insufficient column and the 
bottom row intersect. 
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Level B Rubric 



Language 


Task Fulfillment/Content Development 


Insufficient 


Minimal 


Partial 


Complete 


• variety of simple and complex sentences 

• advanced vocabulary and complex verb 
tenses; few errors in form and usage 

• few if any errors in simple structures; 
some errors in complex structures 






5 

(AL) 


6 

(AH) 


• variety of simple sentences; complex 
sentences usually attempted 

• advanced vocabulary and complex verb 
tenses; some errors in form and usage 

• few errors in simple structures; frequent 
errors in complex structures 




4 

(IH) 


5 

(AL) 


5 

(AL) 


• predominantly simple sentences; some 
complex sentences may be attempted 

• some attempt to use advanced vocabulary 
and complex verb tenses; may be 
frequent errors in form and usage 

• some errors in simple structures; 
frequent errors in complex structures 


3 

(IL) 


4 

(IH) 


4 

(IH) 




• simple sentences only 

• frequent errors in student generated 
language 

• some copying from prompt or text 


3 

(IL) 


3 

(IL) 






• one or two student generated sentences; 
simple structures only 

• frequent errors in student generated 
language 

• several words, phrases, or sentences 
directly copied from prompt or text 


retest 
Level A 









Task Fulfillment/Content Development Definitions 



Insufficient: 

Minimal: 

Partial: 

Complete: 



Few, if any, of the task requirements are fulfilled; ideas are not integrated 
or developed 

Some requirements of the task are fulfilled; ideas are loosely integrated but 
may lack development 

Most requirements of the task are fulfilled; ideas are usually integrated 
and developed 

All requirements of the task are fulfilled; most ideas are effectively 
integrated and developed 
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General Overview 



The listening specifications are divided into two sections: 

• Listening text specifications 

• Listening item specifications 

Each section is organized as follows. First there is an overview page which 
presents the general organization and defines critical terms used in the 
section. Then, specifications are presented including prototypes that 
exemplify the text or item types being defined. A brief description of each 
section is provided below. 

Listening text specifications 

Listening text specifications begin with general guidelines for text 
selection, adaptation, and development. The specifications for the 
following text types abstracted from the Model Standards and validated 
for placement are then presented: Monologue (short, medium, long, 

and extended) and Dialogue (short, medium, long, and extended). 

Listening item specifications 

Listening item specifications are organized by skill. The following 
listening skills were abstracted from the Model Standards for 
placement: 12 Extract Specific Information, Draw Meaning, Extract 
Global Information, and Make Inferences. For each skill, multiple item 
types are specified. 



12Listening skills were reviewed and initially validated for placement by working group 
members. This process led to several recommendations in the categorization and 
definition of listening skills which are reflected in the listening item specifications. The 
existing definitions, however, were not validated by the working group and thus must be 
systematically reviewed by the advisory committee as part of the validation process. 
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Listening Text Specifications 
Overview 

For the purposes of these specifications, listening texts are categorized 
along two primary dimensions: the number of speakers and length of the 
aural input. These two dimensions are critical in specifying the listening 
text t}Tjes and are discussed below. 

Number of speakers 

A listening text is either a monologue, a text spoken by one person, or a 
dialogue, an interaction between two speakers. The number of speakers 
has implications for the nature of the language contained in the text, 
both in terms of sentence-level features such as complexity of syntax and 
degree of formality and in terms of text organization. 

In general, monologues are characterized by planned, formal speech 
with full forms and few pauses or errors. Discourse is connected 
through subordination as well as coordination. Monologues are further 
differentiated by genre, with text types ranging from short 
announcements to lectures or speeches. Lengthy monologues tend to be 
organized like written texts, with clearly identifiable main ideas and a 
coherent progression of ideas. 

With dialogues it is important to make an additional distinction between 
institutional and personal interactions. An institutional interaction is a 
dialogue between two people in clearly defined roles such as: doctor and 
patient, clerk and customer, teacher and student. The content of the 
institutional interaction is clearly related to the roles of the speakers, 
and the interaction is somewhat predictable in that the types of 
questions and answers are restricted to those that typically occur in a 
particular context (doctor’s office, post office, etc.). By contrast, a 
personal interaction is a dialogue between two people who are not in 
specific institutional roles. The content of a personal interaction is 
therefore not tied to the roles of the speakers. It may be related to any 
aspect of daily life and may not follow any particular structure. 

The distinction between personal and institutional interactions has 
implications for the type of language contained in the dialogue. 
Personal dialogues are often characterized by unplanned, informal 
speech, with frequent reduced forms, pauses, and occasional errors. 
Discourse is connected primarily through coordination with little use of 
subordination. Institutional dialogues tend to be somewhat more 
formal than personal dialogues, though they are generally less formal 
than monologues. 

Figure 3 provides a visual representation of language characteristics 
t 3 q)ically found in monologues and dialogues. 





99 



97 



Listening text specifications 



Overview 



Figure 3 

Language Characteristics of Monologues and Dialogues 




Length of aural input 

The length refers to the amount of text the listener hears. For the 
listening text specifications in this document, length is determined by 
the total number of words. The following operational definitions of 
length are used: 



Short: 

Medium: 

Long: 

Elxtended: 



fewer than 50 words 
50-100 words 
101-250 words 
251-350 words 



There are thus eight possible text types based on these two dimensions 
(number of speakers and length): short, medium, long, and extended 

monologues, and short, medium, long, and extended dialogues. 

Figure 4 below summarizes the eight listening text types and provides 
several examples of monologue genres. 
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Figure 4 

Listening Text Types 



Length 


Monologue 
(Example Genres) 


Dialogue 


Short 


Single Sentence 


Short Interaction 


(<50 words) 


Brief Recorded Message 
Announcement 




Medium 


Recorded Message 


Medium Interaction 


(50-100 words) 


Public announcement 
Brief report 




Long 


Speech 


Long Interaction 


(100-250 


Lecture 




words) 


Report 




Extended 


Speech 


Extended Interaction 


(250-350 


Lecture 




words) 


Report 





The text specifications for short, medium, long, and .extended monologues 
and dialogues are organized as follows. First, general guidelines for 
listening text selection, adaptation, and development are presented. Then, 
specifications for each text type are provided. The specifications include the 
following: 

General description 

A brief definition of the text type 

Length 

Monologue: The recommended number of words 
Dialogue: The recommended number of words and turns 

A turn is defined as the uninterrupted speech of a single person. Each 
turn is between one and four clauses in length (no more than 40 words). 

Number of passes 

The number of times the test taker hears the listening text 

A short or medium text is heard one time only. A long text is heard 
twice in its entirety. An extended text is heard once in its entirety and 
then a second time in segments. 
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Number of items per text 

The recommended number of test items associated with the text in an 
operational form of a test 

Note: For a given text, as many items as possible should be drafted and 
pretested in order to generate a sufficient number of acceptable items for 
an operational form of a test. 

Target level 

The Model Standards proficiency levels at which a listening text type is 
targeted 

The target level of a listening text depends on the text length and the 
language in the text. Listening texts associated with more than one 
item should include vocabulary and structures targeted at several 
levels. 

Language is defined as targeted at a given level when test takers at that 
level are presumed to be able to interpret the language, while test takers 
at the next lower level are not. 

Test level 

An indication of whether the text type is appropriate for Level A tests or 
Level B tests 

SkiUs 

The listening skills that can be tested using the text type 
Prototype listening text 

A listening text that exemplifies the text type^^ 

All prototype texts were pilot tested in the form in which they are 
presented and were found to be promising as models for placement test 
development based on content considerations and feedback from 
teachers and students. 

Prototype listening text notes 

Explanatory notes about the prototype text that include a discussion of 
the text vis-a-vis the specified text attributes 



^^All information presented aurally to the test taker is shown in italics. 
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Text Selection^ Adaptatum, and Development 



Listening texts are either (a) selected from existing recordings, (b) adapted 
from existing recordings and re-recorded, or (c) developed for testing 
purposes. The following general guidelines should be adhered to in 
producing all listening texts. 

1. All listening texts should be professional audiotape recordings based on 
materials originally prepared for a general English-speaking audience 
and not for language learning or assessment purposes. 

2. Listening texts should be drawn from general content areas that are 
familiar to adult ESL students given their goals and experiences. 
General topics include but are not limited to shopping, banking, 
housing, health, transportation, current events, and community 
resources. Some common vocational topics such as employment and 
general workplace safety may also be appropriate. 

3. Care must be taken to select topics that will not quickly become dated or 
that critically depend on background knowledge of specific people or 
events. 

4. The language of the speaker(s) should be standard American English 
spoken at a natural rate with standard pronunciation. Speakers can be 
either male or female. If there is more than one speaker, it is 
preferable to have one male and one female. 

5. The appropriateness of a text for a given level depends in part on factors 
such as familiarity of topic, visual aids, and complexity of content, 
vocabulary, or syntax, as specified in the Model Standards. All texts 
must be reviewed by content-area experts familiar with the Model 
Standards to evaluate their appropriateness for use at specific 
proficiency levels. 

6. Permission must be obtained to use a selected or adapted text on an 
operational test. The source of the text must be acknowledged on all test 
forms. 
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7. An adapted text should remain as close to its original format as 
possible, although the following adaptations may be made: 

(a) Brand names, other proper nouns, addresses, and telephone 
numbers should be changed to avoid association with actual people, 
organizations, places, and so on. 

(b) A text may be shortened or edited for simplicity and clarity, or to 
make a given item type possible. On a Level A text, the language 
may be simplified in places, as long as the text retains a natural 
flow. 

(c) Visual stimuli may be added to the test booklet to help orient the test 
taker to the listening text. 

8. A developed text should be based on authentic materials or real-life 
situations. A monologue can be developed based on information 
presented in a written text. For example, a broadcast can be based on 
information found in a newspaper, and a lecture can be based on 
information presented in a textbook. In scripting the text, care must be 
taken to use language appropriate for the genre being developed. A 
dialogue can be developed based on a simulation of a personal or 
institutional situation. It is preferable to have two people role-play the 
situation several times without a written script and base the final 
recording script on the transcripts of the role plays, rather than 
beginning with an invented script. 
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Short Monologue 



General description 

A short monologue consists of one to four sentences spoken by a single 
speaker in which no more than two or three pieces of specific 
information are presented. Short monologues include single sentence 
descriptions, brief announcements, and brief telephone messages. 

Length 

Fewer than 50 words 

Number of passes 
One 

Number of items per text 
One 

Target level 

Beginning low through intermediate low 
Test level 

Appropriate for Level A tests and as a warm-up only for Level B tests 
Skills 

Extract specific information 
Prototype text 

Thank you for calling the Metro Bus. All of our information staff are busy 
at the moment. Please hold, and your call will be answered as soon as 
possible. Thank you. 



Prototype text notes 

The listening text consists of three sentences (24 words) and represents 
a typical outgoing message heard on an institutional answering 
machine. The example text is targeted primarily at the beginning levels 
and is thus appropriate for Level A tests and as a warm-up for Level B 
tests. 
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Medium Monologue 



General description 

A medium monologue consists of five to ten sentences spoken by a single 
speaker in which at least six pieces of specific information are 
presented. Medium monologues include telephone messages, public 
announcements, and brief television and radio broadcasts. 

Length 

Between 50 and 100 words 

Number of passes 
One 

Number of items per text 

Between three and six (fewer items than pieces of information) 

Target level 

Beginning high through advanced high 
Test level 

Appropriate for both Level A and Level B tests 
Skills 

Extract specific information 
Prototype text 

Hello, you have reached the Children’s Zoo. The zoo is located in Adams 
Park at the intersection of Pacific and Main. We are open daily from 
ten a.m. to five p.m. The zoo is closed on Christmas Day only. Admission 
prices are: adults ages 13 and over, eight dollars; children ages two through 
12, three dollars; senior citizens ages 65 and over, five dollars; and 
children under two are admitted free. 

Prototype text notes 

The listening text consists of five sentences (71 words) and represents a 
typical outgoing message heard on an institutional answering machine. 
It contains several pieces of specific information including location, 
hours, and admission prices. The example text is targeted primarily at 
the intermediate levels and is thus appropriate for both Level A and 
Level B tests. 
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Long Monologue 



General description 

A long monologue consists of one to three paragraphs spoken by a single 
speaker in which several pieces of information are presented. Long 
monologues include short speeches, lectures, and broadcasts. 

Length 

Between 100 and 250 words 

Number of passes 
Two 

Number of items per text 
Between four and six 

Target level 

Intermediate low through advanced high 
Test level 

Appropriate for both Level A and Level B tests 
Skills 

Extract specific information 
Draw meaning 
Make inferences 

Prototype text 

A long monologue was not developed as part of the prototyping effort. To 
complete this specification, a long monologue that could serve as a 
prototype text should be drafted and pilot tested. 
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Elxtended Monologue 



General description 

An extended monologue consists of four to eight related paragraphs (250 
to 350 total words), which for testing purposes is divided into two to three 
segments. Each segment consists of two to four paragraphs (50 to 200 
words). Care should be taken to segment the text at points that 
represent a natural break so that content unity is maintained. 

An extended monologue is heard once in its entirety and the test taker 
answers one global question about the text. Then the monologue is heard 
a second time in segments. After each segment, the test taker answers 
between one and four questions about the segment just heard. These 
questions may require the test taker to extract specific information, 
draw meaning, or make inferences. 

Figure 5 below summarizes this information. 

Figure 5 

Characteristics of Extended Monologues 





Entire Monologue 


Each Segment 


Length 


Between 250 and 350 words 


Between 50 and 200 words 


Number of 
passes 


One 


One 


Number of 
items 


One 


Level A: One 

Level B: No more than four 


Skills 


Extract global information 


Extract specific information 
Draw meaning 
Make inferences 



Target level 

Intermediate low through advanced high 
Test level 

Appropriate for both Level A and Level B tests 




1 0 6 
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Prototype text 



Philip Vera Cruz passed away in Bakersfield, California yesterday. He 
was 89 years old. Vera Cruz was a migrant worker who became a political 
activist and labor organizer in California. He is best remembered as an 
organizer of Filipino workers and co-founder of the United Farm Workers 
Union. 

Vera Cruz immigrated from the Philippines when he was 22 years old, 
hoping to get a good job and an education. However, like many other 
Filipino immigrants who came to the United States in the 1920s, he was only 
able to find low-paying jobs in factories, canneries, and fields. Because 
these jobs were short-term, Vera Cruz had to move frequently in order to 
find work: cutting boxes in a factory in Washington, canning fish in 
Alaska, and picking grapes in California. 



Vera Cruz was paid very little, yet he sent most of the money he earned to his 
family in the Philippines. He never married, fearing that if he had his own 
family to take care of in the States, he would not be able to help his family in 
the Philippines. 

Vera Cruz attended college briefly, but his limited income and commitment 
to supporting his family made it impossible for him to realize his dream of 
becoming a lawyer. He did, however, succeed in making education 
possible for his family: he helped his sister, his brother, and all his brothers 
children go to college. 

Vera Cruz was angered by the low wages, long hours, and poor living 
standards of migrant workers, and in the mid-1960s helped organize the 
Filipino workers to fight for improved working conditions. In 1965 he 
joined forces with Cesar Chavez to establish the United Farm Workers 
Union, where he served as vice president until 1977. Because of his deep 
concern for workers and anger at the way they were treated, Vera Cruz 
dedicated his life to fighting for social justice and equality. He will be 
remembered as a great leader and activist. 

Prototype text notes 

The listening text is a news report consisting of five paragraphs (330 
total words) that retells important events in a famous person’s life. The 
example text is targeted primarily at the advanced levels and is thus 
appropriate for Level B tests. There are two segments of 132 and 197 
words respectively. 
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Short Dialogue 



General description 

A short dialogue is a brief personal or institutional interaction between 
two speakers containing fewer than 50 words and a maximum of three 
turns. 

Length 

Words: Fewer than 50 
Turns: Between two and three 

Number of passes 
One 

Target level 

Beginning low through beginning high 
Test level 

Appropriate for Level A tests only 

Number of items per text 
One 

Skills 

Extract specific information 

Prototype text 

Speaker 1: What’s wrong'? 

Speaker 2: I can’t find my sunglasses. 

Prototype text notes 

The listening text represents a simple personal exchange. There are 
two turns and seven words. The example text is targeted at the 
beginning levels and is appropriate for Level A tests only. 
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Listening text specifications 



Medium dialogue 



Medium Dialogue 



General description 

A medium dialogue is a personal or institutional interaction between 
two speakers containing 50 to 100 words and 4 to 10 turns. 

Length 

Words: Between 50 and 100 
Turns: Between 4 and 10 

Number of passes 
One 

Target level 

Intermediate low through advanced high 
Test level 

Appropriate for both Level A and Level B tests 

Number of items per text 
Between three and five 

Skills 

Extract specific information 
Draw meaning 

Prototype text 

A medium dialogue was not developed as part of the prototyping effort. 
To complete this specification, a medium dialogue that could serve as a 
prototype text should be drafted and pilot tested. 



Listening text specifications 



Long dialogue 



Long Dialogue 



General description 

A long dialogue is a personal or institutional interaction between two 
speakers containing 100 to 250 words and 6 to 20 turns. 

Length 

Words: Between 100 and 250 
Turns: Between 6 and 20 

Number of passes 

Two (each time in its entirety) 

Target level 

Intermediate low through advanced high 
Test level 

Appropriate for both Level A and Level B tests 

Number of items per text 
Between three and five 

Skills 

Extract specific information 
Draw meaning 
Make inferences 



O 
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Listening text specifications 



Long dialogue 



Prototype text 

Doctor: 

Mother: 

Doctor: 

Mother: 

Doctor: 

Mother: 

Doctor: 



Mother: 

Doctor: 

Mother: 

Doctor: 

Mother: 



Everything seems fine, but her throat's a little red. I want to take a 
throat culture to make sure she doesn't have an infection. 

Okay. So, um, how long will it take for the test results to come 
back? 

We should have the results in two days. You should call us then to 
see if they're ready. 

All right. So, she might have a throat infection. You're going to do 
a test and I should call you back in two days to see what the results 
are. 

Uh’huh. And if she does have a throat infection I'll prescribe some 
antibiotics at that time. 

Okay. What should I do in the meantime? 

You can get some children's Tylenol at the pharmacy. Give it to 
her every four to six hours until her fever has gone down. She 
should continue to drink a lot of liquids and get plenty of rest 
Okay. How long should I keep her out of school? 

You can let her go back to school when her temperature is normal 
and she's feeling better. 

Okay. Should I schedule a follow-up appointment? 

I don't think it's necessary at this time, but if her symptoms don't 
improve in a few days, I would like to see her again. 

All right. That sounds good. Thank you very much. Doctor Li. 



Prototype text notes 

The text is an institutional interaction between a doctor and a patient’s 
mother. There are 12 turns and 221 words. The exgimple text is targeted 
primarily at the intermediate levels and is thus appropriate for both 
Level A and Level B tests. 
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Listening text specifications 



Extended dialogue 



Extended Dialogue 



General description 

An extended dialogue is a personal or institutional interaction between 
two speakers consisting of 250 to 350 words (between 25 and 40 turns), 
which is divided into two to four segments consisting of 50 to 150 words 
(between 5 and 20 turns). Care should be taken to segment the text at 
points that represent a natural break so that content unity is 
maintained. 

An extended dialogue is heard once in its entirety and the test taker 
answers one global question about the interaction. Then the text is heard 
a second time in segments. After each segment, the test taker answers 
between one and three questions about the segment heard. These 
questions may require the test taker to extract specific information, 
draw meaning, or make inferences. 

Figure 6 below summarizes this information. 

Figure 6 

Characteristics of Extended Dialogues 





Entire Dialogue 


E^h Segment 


Length 


Between 250 and 350 words 
Between 25 and 40 turns 


Between 50 and 150 words 
Between 5 and 20 turns 


Number of 
passes 


One 


One 


Number of 
items 


One 


Level A: One 

Level B: No more than three 


Skills 


Extract global information 


Extract specific information 
Draw meaning 
Make inferences 



Target level 

Intermediate low through advanced high 
Test level 

Appropriate for both Level A and Level B tests 
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Listening text specifications 



Extended dialogue 



Prototype text 



Doctor: 

Mother: 

Doctor: 

Mother: 

Doctor: 

Mother: 

Doctor: 

Mother: 

Doctor: 

Mother: 

Doctor: 

Mother: 

Doctor: 

Mother: 

Doctor: 

Mother: 



Good morning, Fm Dr. Li. 

Hi Dr. Li, Fm Mrs. Johnson. 

So, what brings you here today? 

Well, my daughter hasn't been feeling well and Fm a little concerned. 
Okay. Well, what's your daughter's name? 

Natalie. 

And how old is she? 

She's five years old. 

Okay. And what are her symptoms? 

She has a sore throat and a fever. 

How long has she been sick? 

Well, let's see. Her sore throat started a few days ago and last night 
when I took her temperature she had a slight fever. 

All right. Has she been eating normally? 

More or less, but it's been a little hard for her to eat. Her throat is really 
sore and it hurts when she swallows. 

How has she been acting? 

A little tired. She's been sleeping more than she usually does. 



Doctor: 

Mother: 

Doctor: 

Mother: 

Doctor: 

Mother: 

Doctor: 

Mother: 

Doctor: 

Mother: 



Okay. What have you been doing for Natalie? 

Well, we kept her home from school yesterday and made sure she got 
plenty of rest. 

Anything else? 

We gave her a lot of juice and water to drink. 

Have you given her any medicine? 

No, not yet. 

Okay. Are any of Natalie's friends at school sick? 

Not that I know of 

Is anyone else in the family sick? 

Well, my son was home sick last week. 



Doctor: I see. Is she generally a healthy child? 

Mother: Yes. 

Doctor: Okay. Is she allergic to any medicines? 

Mother: Not that I know of 

Doctor: Has she had any surgeries in the past? 

Mother: No. 

Doctor: Okay. Any medical problems in your family that I should be aware 
Mother: No. 

Doctor: Okay. Fm going to examine Natalie now. 



of? 



Prototype text notes 

The listening text is an institutional interaction that takes place at a 
doctor’s office. There are 35 turns and a total of 257 words. The text is 
divided into three segments of 131 words, 75 words, and 51 words 
respectively. The example text is targeted primarily at the intermediate 
levels and is thus appropriate for both Level A and Level B tests. 



Listening item specifications 



Overview 



Listening Item Specifications 
Overview 

Item specifications are provided for the listening skills below. 

• Extract specific information (ES) 

• Draw meaning (DM-L) 

• Extract global information (EG) 

• Make inferences (MI-L) 

For each skill, an introductory page provides the following information: 
General description 

A brief definition of the skill as operationalized in item types 
Text type 

The aural stimulus (monologue or dialogue) on which listening items 
are based 

Target level 

The Model Standards proficiency levels at which items testing the skill 
are targeted 

An item is defined as being targeted at a given level when test takers at 
that level are presumed to be able to answer the item correctly while test 
takers at the next lower level are not. 

The target level of an item is dependent on both the skill being tested and 
the language that must be interpreted in the aural stimulus in order to 
answer the item. 

Test level 

An indication of whether the skill is essential or optional for placement 
on Level A tests and Level B tests 

Note: Although a skill may be essential for both Level A and Level B 
tests, not all item types are appropriate for both levels. (See Test Level 
under individual item types for level appropriateness.) 

Item types 

A brief description of each item type testing the s kill 

\J 
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Listening item specifications 



Overview 



The introductory page is followed by specifications for each item type 
associated with that skill. The specifications include the following 
information: 

Item format 

The format of the item type (fill in the blank, matching, multiple choice, 
select number correct, sequencing, unique answer) 

Test level 

An indication of whether the item type is appropriate for Level A and 
Level B tests 

Stimulus attributes 

The characteristics of the input to the test taker, which include: 

(a) specific information about the aural stimulus and, where 
appropriate, the visual stimulus 

(b) a detailed description of the item stem 

(c) an indication of where target-level language is found in the 
stimulus 

Response attributes 

The characteristics of the response 

For constructed response items, a detailed description of the expected 
response is provided. 

For selected response items, the following are specified: 

(a) number of response options 

(b) characteristics of the key (correct option) 

(c) characteristics of the distractors (incorrect options) 

(d) language characteristics of the response options, where appropriate 

Prototype item or item set 

An item or item set that exemplifies the item type including the aural 
stimulus and information found in the test booklet 

The aural stimulus indicates what the test taker hears including the 
directions and, where appropriate, the item stem. All information 
presented aiirally to the test taker is shown in italics. 

Test booklet information includes the directions and, where appropriate, 
the item stem, the response options, and the visual stim\ilus. 



Listening item specifications 



Overview 



Unless otherwise noted, all prototype items were pilot tested in the form 
in which they are presented and were found to be promising as models 
for placement test development based on statistical performance and 
content considerations. 

The key is indicated by an asterisk (*) for selected response items. 

Note: Directions are provided in both aural and written form. In pilot 
testing, written directions were as brief as possible to limit the amount 
of reading required and more extensive directions were provided 
aurally. Once whole tests are constructed this issue will need to be 
revisited to determine the most effective balance. 

Prototype item notes 

Explanatory notes about the prototype item include the following: 

(a) a discussion of the item vis-a-vis the specified stimulus and 
response attributes, including the target-level language 

(b) examples of acceptable and unacceptable responses for constructed 
response items 

(c) the estimated range and discrimination point for the item or item 
set based on pilot testing results 

The estimated range indicates the proficiency levels at which the 
item seems most appropriate in terms of content and performance. 
The discrimination point indicates the two adjacent levels where the 
item seems to best differentiate student performance. (See Appendix 
B for criteria used to determine both estimated range and 
discrimination point.) 

Note: Some prototype items were modified based on pilot testing 
results. For these items, the estimated range and discrimination 
point are not available because they have not been pilot tested in their 
current form. 

Additional notes about the item types are provided at the end of each skill 
section. These notes include supplementary information about the item 
types from pre- and pilot testing, such as considerations for alternative 
item formats and variations in wording. 
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Listening item specifications 



Extract specific information 






Extract Specific Information (ES) 



General description 

The test taker listens for and identifies specific information in the form 

of a word or phrase within the text. 

Text type 

Short or medium monologue or dialogue 

Segment of an extended monologue or dialogue 

Target level 

Beginning low through advanced high 

Test level 

Level A: essential 

Level B: essential 

Item types 

ESI The test taker selects one of four pictures that corresponds to an 
object heard in a short text 

ES2 The test taker selects one of two numbers that corresponds to a 
piece of numerical information presented in a short text 

E1S3 The test taker matches information presented in a series of short 
texts to scenes depicting the information described 

ES4 The test taker selects one of four specific actions that corresponds 
to an action stated in a short text 

ESS The test taker writes down specific information heard in a 
medium text 

E1S6 The test taker selects two of five pictures that answer a factual 
question about a long text or a segment of an extended text 

ES7 The test taker selects three of six options that answer a factual 
question about a long text or a segment of an extended text 

ESS The test taker writes down a word or phrase that answers a 
factual question about a long text or a segment of an extended text 





121 



117 



Listening item specifications 



Extract specific information 



Item Type ESI 



Item format 

Multiple choice (picture) 

Test level 

Appropriate for Level A tests, as a warm-up only 
Stimulus attributes 

The aural stimulus is a short monologue or dialogue in which a single 
common object is mentioned once. The aural stimulus is heard one 
time only. 

The aural stimulus contains target-level language. 

Response attributes 

A set of four pictures depicting common objects is printed next to an 
item number. The objects must be plausible in that any of them could be 
used in the text in place of the object mentioned. The test taker must 
circle the object referred to in the text. 



Listening item specifications 



Extract specific information 






Prototype item 
Aural stimulus 

Listen to the conversation. What are the people talking about? Circle the correct 
picture. 

Number one 

Speaker 1: What’s wrong? 

Speaker 2: I can’t find my sunglasses. 

Test booklet 



Circle the correct picture. 



1 . 







Prototype item notes 

The aural stimulus is a short dialogue. The test taker must recognize 
the word “sunglasses” and match it to the correct picture. The 
distractors are other objects that could be misplaced. 

Estimated range: Easy for the population Discrimination point: BL- 
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Listening item specifications 



Extract specific information 






Item Type ES2 



Item format 

Multiple choice 

Test level 

Appropriate for Level A tests only 
Stimulus attributes 

The aural stimulus is a short monologue or dialogue containing a 
single piece of numerical information such as a telephone number or 
address. The aural stimulus is heard one time only. 

The aural stimulus contains target-level language. 

Response attributes 

Two series of numbers are printed next to an item number. The key is 
the number mentioned in the text. The distractor differs from the key in 
that either one or two pairs of numbers from the key are transposed. 
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Extract specific information 



I 



Prototype item 
Aural stimulus 

Listen to the telephone message and circle the phone number you hear. 
Number one 

Hi. You’ve reached 452-2864. No one’s here to get your call right now, but 
leave us a message and we’ll get back to you as soon as we can. Thanks for 
calling. Bye. 



Test booklet 



Circle the phone number you hear. 



1. 452-2864* 542-8264 



Prototype item notes 

The aural stimulus, a short monologue, is a brief telephone message. 
The test taker must identify the telephone number heard in the 
message. In the distractor, two pairs of numbers have been transposed 
(45 and 54; 28 and 82). 

Estimated range: BL-IL Discrimination point: BL- 
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Listening item specifications 



Extract specific information 



Item Type ESS 



Item format 
Matching 

Test level 

Appropriate for Level A tests only 
Stimulus attributes 

The aural stimulus is a series of short monologues or dialogues 
describing an action taking place within a visual stimulus. Before each 
monologue or dialogue is heard, the item number associated with it is 
stated. There are at least three short monologues or dialogues per 
visual. 

The visual stimulus (drawing or photograph) is a scene showing several 
people engaged in different activities. Each person or group of people 
engaged in the same activity is marked with a letter (a, b, c, etc.). There 
are more scenes marked than short monologues or dialogues heard. 

The aural stimulus contains target-level language. 

Note: An example is provided to orient the test taker to the task. 

Response attributes 

A series of blanks corresponding to the number of monologues or 
dialogues is provided in the test booklet. The test taker must match the 
activity described in the aural stimulus with the letter corresponding to 
the scene depicting the activity by writing down the letter of the activity 
being described in the blank next to the item number. 
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Listening item specifications 



Extract specific information 



Prototype item set 
Aural stimulus 

Look at the drawing of the park. There are letters next to some of the 
pictures. Listen to the sentence and write the correct letter. 



K^Y 

d 


Example: 


He's reading the newspaper. 


c 


Number 1: 


She's feeding the birds. 


b 


Number 2: 


They're in love. 


e 


Number 3: 


She's standing near the fountain. 


a 


Number 4: 


There's a bird in the tree. 


Test booklet 





Write the correct letter next to the number. 




Example: 



1. 2. 3. 4. 



Prototype item notes 

The aural stimulus, a series of short monologues, consists of a five 
single sentences that describe scenes depicted in the drawing of the 
park. The test taker must match the scene described in the sentence to 
the letter corresponding to the correct group of people or objects. 

Estimated range: BL-IL (1,2,3,4) Discrimination point: BL/BH (1,2,4) 

BH/IL (3) 



Listening item specifications 



Extract specific information 



Item Type ES4 



Item format 

Multiple choice 

Test level 

Appropriate for Level A tests and as a warm-up for Level B tests 
Stimulus attributes 

The aural stimulus is a short monologue or dialogue containing an 
instruction for a specific action to take. 

The aural stimulus contains target-level language. 

Response attributes 

A set of four actions (single words or phrases) is printed in the test 
booklet. The key is the action referred to in the aural stimulus. The 
three distractors are other plausible actions the listener could take. 

The response options use words or phrases different from those heard in 
the aural stimulus, where possible, and contain language at a lower 
level than the target level. 



Listening item specifications 



Extract specific information 



) 



Prototype item 
Aural stimulus 

Listen to the message. What does the message tell you to do"? Check the 
correct answer. 

Thank you for calling the Metro Bus. All of our information staff are busy 
at the moment. Please hold, and your call will be answered as soon as 
possible. Thank you. 



Test booklet 



Check (V) the correct answer. 



wait* 

call again later 
leave a message 
call a different number 



Prototype item notes 

The aural stimulus, a short monologue, is a brief recorded telephone 
message which asks the listener to wait to be helped. The test taker 
must extract this information to find the correct answer. The 
distractors are plausible actions typically heard in telephone messages. 

Estimated range: BH-AL Discrimination point: IL/IH 
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Listening item specifications 



Extract specific information 



Item Type ES5 



Item format 

Fill in the blank 

Test level 

Appropriate for Level A tests and as a warm-up for Level B tests 
Stimulus attributes 

The aural stimulus is a medium monologue or dialogue containing 
several pieces of specific information consisting of a simple number, 
word, or phrase. The test booklet contains a visual stimulus which 
presents some of the information from the aural stimulus and provides 
blanks (between three and six) where information is to be filled in. 

The aural stimulus contains target-level language. The visual stimulus 
contains words and phrases at a lower level. 

Note: There should be fewer blanks than pieces of information. 
Response attributes 

A blank is provided for each piece of missing information. The test 
taker must write the missing information in the blank. 



Listening item specifications 



Extract specific information 



Prototype item set 
Aural stimulus 

You will hear a telephone message about the Children’s Zoo. You will hear 
the message one time only. Look at the information about the zoo. The 
hours and prices are missing. (Pause 15 seconds.) Now listen to the 
message and fill in the blanks. 

Hello, you have reached the Children’s Zoo. The zoo is located in Adams 
Park at the intersection of Pacific and Main. We are open daily from 
ten a.m. to five p.m. The zoo is closed on Christmas Day only. Admission 
prices are: adults ages 13 and over, eight dollars; children ages two through 
12, three dollars; senior citizens ages 65 and over, five dollars; and 
children under two are admitted free. 

Test booklet 



Fill in the blanks. 



Children ’s Zoo^ 


Location: Adams Park 


Hmirfi* n m to _ 

1 


p.m. 

2 


Closed: Christmas Day 


Prices: 


Price 


Adults 


$8 


Children 


3 


Senior citizens 


4 


Under 2 years 






5 



Prototype item notes 

The aural stimulus, a medium monologue, is a recorded message 
containing several pieces of specific information. The visual stimulus is 
a facsimile of a note card containing blanks for the missing 
information. The test taker must listen for the missing hours and 
prices and write the information in the blanks provided. 

Estimated range: BL-IH (1,2,3,4) Discrimination point: BL/BH (2,3,4) 

BH-IH (5) BH/IL (1, 5) 
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Listening item specifications 



Extract specific information 



Item Type ES6 



Item format 

Select two correct (pictures) 

Test level 

Appropriate for Level A tests only 
Stimulus attributes 

The aural stimulus is a long monologue or dialogue or a segment of an 
extended monologue or dialogue. The text refers to two persons, objects, 
actions, or situations that can be concretely depicted in drawings or 
photographs. A long text is heard two times. A segment is heard one 
time after the extended text is heard in its entirety. 

The item stem consists of a factual question about the text requiring the 
test taker to identify two correct answers. The item stem is heard on the 
tape and is written in the test booklet. 

The segment contains target-level language. The item stem contains 
language at a lower level. 

Response attributes 

A set of five pictures depicting persons, objects, actions, or situations 
follows the item stem. Underneath each picture is a letter. The test 
taker must circle the letter of the two correct answers. The two keys are 
the pictures representing the persons, objects, actions, or situations 
mentioned in the segment. The three distractors represent persons, 
objects, actions, or situations that are plausibly related to the content of 
the segment but are not mentioned in the segment. 
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Listening item specifications 



Extract specific information 






Prototype item 
Aural stimulus 

Listen to the conversation and circle the two correct answers. 
Whafs wrong with Natalie? 



Doctor: 

Mother: 

Doctor: 

Mother: 

Doctor: 

Mother: 

Doctor: 

Mother: 



Okay. What are Natalie's symptoms? 

She has a sore throat and a fever. 

How long has she been sick? 

Well, let's see. Her sore throat started a few days ago and last 
night when I took her temperature, she had a slight fever. 

All right. Has she been eating normally? 

More or less, but it's been a little hard for her to eat. Her throat is 
really sore and it hurts when she swallows. 

How has she been acting? 

A little tired. She's been sleeping more than she usually does. 



Test booklet 



Circle the two (2) correct answers. 



What's wrong with Natalie? 




c 




b* 






d* 

Prototype item notes 

The aural stimulus is a segment of an extended institutional dialogue 
between a doctor and a child's mother. The test taker must listen to the 
segment to find the correct answers "sore throat” and “fever” which are 
repeated several times. The three distractors (backache, stomach ache, 
and broken arm) are not mentioned in the segment. 

Estimated range: BL-IL Discrimination point: BUEH. 
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Listening item specifications 



Extract specific information 



Item Type ES7 



Item format 

Select three correct 

Test level 

Appropriate for Level B tests only 
Stimulus attributes 

The aural stimulus is a long monologue or dialogue or a segment of an 
extended monologue or dialogue. The text refers to three objects, actions 
or situations that can be described in a single word or phrase. A long 
text is heard two times. A segment is heard one time after the extended 
text is heard in its entirety. 

The item stem consists of a factual question about the text requiring the 
test taker to identify three correct answers. 

The segment contains target-level language. The item stem contains 
language at a lower level. 

Response attributes 

A set of six words or phrases appears below the question. Each word or 
phrase is preceded by a blank. T^e test taker must check the blank next 
to each of the three correct answers. The three keys are the words or 
phrases representing the three objects, actions, or situations mentioned 
in the segment. The keys may use the exact words found in the text or 
synonyms. The three distractors represent objects, actions, or 
situations that plausibly relate to the content of the segment but are not 
mentioned in the segment. The response options contain target-level 
language or language at the lower level. 



Listening item specifications 



Extract specific information 




Prototype item 
Aural stimulus 

Read the question. Now, listen to the third part of the conversation and 
check the three correct answers. 



Doctor: 

Mother: 

Doctor: 

Mother: 

Doctor: 

Mother: 

Doctor: 

Mother: 

Doctor: 



I see. Is she generally a healthy child? 

Yes, 

Okay, Is she allergic to any medicines? 

Not that I know of. 

Has she had any surgeries in the past? 

No, 

Okay, Any medical problems in your family that I should be 
aware of? 

No, 

Okay, Tm going to examine Natalie now. 



Test booklet 



Check (V) the three (3) correct answers. 



What information does Dr. Li ask for? 

x-rays 

allergies* 

past surgeries* 

immunizations 

health insurance 

family medical problems* 

Prototype item notes 

The aural stimulus is a segment of an extended institutional dialogue 
between a doctor and a child's mother. The test taker must listen to the 
questions the doctor asks and indicate the correct answers. The three 
keys incorporate words and phrases heard in the dialogue. The three 
distractors are related to health care, but are not mentioned in the 
segment. 

Estimated range: IL-AH Discrimination point: IL/IH 
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Listening item specifications 



Extract specific information 



Item Type ESS 



Item format 

Unique answer 

Test level 

Appropriate for both Level A and Level B tests 
Stimulus cUtributes 

The aural stimulus is a long monologue or dialogue or a segment of an 
extended monologue or dialogue. The text contains several pieces of 
information which can be described in a single word or phrase. A long 
text is heard two times. A segment is heard one time after the extended 
text is heard in its entirety. 

The item stem consists of a question asking for a piece of alphabetic or 
non-alphabetic information presented in the text. The question 
ordinarily begins with who, what, when, where, how much, or how 
many. 

The segment contains target-level language. The item stem contains 
language at a lower level. 

Response attributes 

The test taker must write down the correct answer, which consists of a 
number, word, or phrase explicitly stated in the text. The test taker is 
not penalized for minor misspellings or grammatical errors as long as 
the meaning is clear. 
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Listening item specifications 



Extract specific information 



Prototype item 
Aural stimulus 

Listen to the first part of the news report and answer the question, 

Philip Vera Cruz passed away in Bakersfield, California yesterday. He 
was 89 years old, Vera Cruz was a migrant worker who became a political 
activist and labor organizer in California, He is best remembered as an 
organizer of Filipino workers and co-founder of the United Farm Workers 
Union, 

Vera Cruz immigrated from the Philippines when he was 22 years old, 
hoping to get a good job and an education. However, like many other 
Filipino immigrants who came to the United States in the 1920s, he was only 
able to find low-paying jobs in factories, canneries, and fields. Because 
these jobs were short-term, Vera Cruz had to move frequently in order to 
find work: cutting boxes in a factory in Washington, canning fish in 
Alaska, and picking grapes in California, 



Test booklet 



Answer the question. 



What happened to Vera Cruz yesterday? 



Prototype item notes 

The aural stimulus, a segment of an extended monologue, is the first 
part of a news report. The test taker must listen for a specific event and 
write down the correct answer. Example responses appear below. 



Acceptable: He died; He passed away yesterday; Past avay (minor 

spelling error); He dead (minor grammatical error) 

Unacceptable: Bakersfield; 89 years old; moved to California; pass 
work 



Estimated range: IH-AH Discrimination point: AL/AH 



ERIC 



137 



133 



Listening item specifications 



Extract specific information 



ES Notes 



1. A variant of item type ES4 which asked the test taker to identify a place 
or organization associated with a short monologue or dialogue was also 
pilot tested. The variation proved to be somewhat easier for test takers 
because the response options usually contained the same word or 
phrase heard in the text. Identifying an action, as in item type ES4, was 
more challenging in that the response options often incorporated words 
or phrases synonymous to but different from those heard in the text. 
This is an example of how modifying the content can affect the difficulty 
of an item. 

2. Item type ESS was pretested with an additional piece of missing 
information. The missing information was non-numeric and consisted 
of the two-word phrase, “Christmas Day.” Pretesting results indicated 
that test takers needed too much time to write down the missing 
information which interfered with their ability to listen to the rest of the 
text and fill in the remaining missing information. These results 
suggest that it is best to request information that is either numeric or a 
short, familiar word or phrase when using this item type. 

3. Scoring Prototype ESS was complicated by the wide range of responses 
that test takers provided due in part to several spelling errors in the 
phrase “passed away.” The range in errors underscored the complexity 
of assessing listening in an open-ended format in that the test taker can 
transcribe information without necessarily understanding the 
information retrieved. Although this was a concern, the scoring 
protocol was generous rather than restrictive in determining acceptable 
and unacceptable responses because a transcribed response indicates 
that the test taker understood the question and was able to isolate the 
portion of the text that addressed the issue. Pilot testing showed that 
developing a scoring protocol for item type ESS requires systematic 
review of the responses. Even so, the item type is recommended for 
operational test because of its usefulness in discriminating at the upper 
levels. 



Listening item specifications 



Draw meaning 



) 



Draw Meaning (DML) 



General description 

The test taker will listen to a text and interpret an idea expressed in the 
text. 

Drawing meaning differs from extracting specific information in that 
the listener cannot simply listen for a specific word or phrase from a 
stream of speech to answer an item. Rather, the listener must be able to 
construct meaning from what is said, which involves interpreting one 
or more of the following: 

• vocabulary and sentence structure, including reduced forms 

• cohesive devices, including pronoun reference, ellipses, and 
lexical cohesion 

• intonation and stress 
Text type 

Medium or long monologue or dialogue 
Segment of extended monologue or dialogue 

Target level 

Intermediate low through advanced high 

Test level 

Level A: essential 
Level B: essential 

Item types 

In item type DMLl, the test taker selects one of four pictxmes that depicts 
the answer to a factual question about a text. In item type DML2, the 
test taker selects one of four responses to a factual question about a text. 
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Listening item specifications 



Draw meaning 



Item Type DMLl 



Item format 

Multiple choice (picture) 

Test level 

Appropriate for Level A tests only 
Stimulus attributes 

The aural stimulus is a long monologue or dialogue or a segment of an 
extended monologue or dialogue. The text contains information about 
persons, objects, actions, or situations that can be concretely depicted in 
drawings or photographs. A long text is heard two times. A segment is 
heard one time after the extended text is heard in its entirety. 

The item stem consists of a factual question about the text. 

The aural stimulus contains target-level language. The item stem 
contains language at a lower level. 

Response attributes 

A set of four pictures depicting persons, objects, actions, or situations 
follows the item stem. Underneath each picture is a letter. The test 
taker must circle the letter of the correct answer. The key is a picture 
representing the person, object, action, or situation mentioned in the 
segment. The three distractors represent persons, objects, actions, or 
situations that plausibly relate to the content of the segment and could be 
described using words or phrases heard in the text but are inaccurate 
answers to the question. 



Listening item specifications 



Draw meaning 



; 



Prototype item 
Aural stimulus 

Listen to the conversation and circle the correct answer. 

What did Natalie do yesterday? 

Doctor: Okay. What have you been doing for Natalie? 

Mother: Well, we kept her home from school yesterday and made sure she 
got plenty of rest. 

Doctor: Anything else? 

Mother: We gave her a lot of juice and water to drink. 

Doctor: Have you given her any medicine? 

Mother: No, not yet. 

Test booklet 



Circle the correct answer. 



What did Natalie do yesterday? 






Prototype item notes 

The aural stimulus is a segment of an extended institutional dialogue 
between a doctor and a child’s mother. The test taker must interpret 
several phrases in the segment to answer the question. The three 
distractors play on words and phrases heard in the segment and relate 
to the content but are incorrect answers to the questions. 

Note: Prototype DMLl was modified based on pilot testing results and 
thus the estimated range and discrimination point are not available. 
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Listening item specifications 



Draw meaning 



Item Type DML2 



Item format 

Multiple choice 

Test level 

Appropriate for Level B tests only 
Stimulus attributes 

The aural stimulus is a long monologue or dialogue or a segment of an 
extended monologue or dialogue. The text contains information about 
persons, objects, actions, or situations that can be concretely depicted in 
drawings or photographs. A long text is heard two times. A segment is 
heard one time after the extended text is heard in its entirety. 

The item stem consists of a factual question about the text. 

The aural stimulus contains target-level language. The item stem 
contains language at a lower level. 

Response attributes 

A set of four single-phrase or single-sentence options follows each stem. 
The response options contain language at a lower level than the target 
level. The test taker must circle the letter of the correct answer. The key 
is a phrase or sentence answering the question using words other than 
those found in the text. 

The three distractors are grammatically accurate and plausibly relate to 
the content of the text but are inaccurate in that they are either 
unsupported or contradicted by information in the text. At least one of 
the distractors incorporates words or phrases from the text. 
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Listening item specifications 



Draw meaning 



Prototype item 



Aural stimulus 

Now you will hear another conversation between Dr. Li and Natalie's mother. 
You will hear the conversation two times. Listen to the conversation and circle 
the letter of the correct answer. 



Doctor: 

Mother: 

Doctor: 

Mother: 

Doctor: 

Mother: 

Doctor: 



Mother: 

Doctor: 

Mother: 

Doctor: 

Mother: 

Doctor: 

Mother: 



Everything seems fine, but her throat's a little red. I want to take a 
throat culture to make sure she doesn't have an infection. 

Okay. So, um, how long will it take for the test results to come back? 
We should have the results in two days. You should call us then to see 
if they're ready. 

All right. So, she might have a throat infection. You're going to do a 
test and I should call you back in two days to see what the results are. 
Uh-huh. And if she does have a throat infection I'll prescribe some 
antibiotics at that time. 

Okay. What should I do in the meantime? 

You can get some children's Tylenol at the pharmacy. Give it to her 
every four to six hours until her fever has gone down. She should 
continue to drink a lot of liquids and get plenty of rest. 

Okay. How long should I keep her out of school? 

It's best to keep her home for the next few days. In fact. I'd like to see 
her again before you send her back to school. 

Okay. So should I schedule a follow-up appointment? 

Yes, you can do that on your way out. 

Fine. By the way, is it all right for her to be around her brother? 

Yes, as long as she washes her hands frequently and they don't share 
utensils or eat off the same plate. 

All right. That sounds good. Thank you very much Doctor Li. 



Test booklet 



Circle the letter of the correct answer. 



What does Dr. Li say about Natalie? 

a. She needs antibiotics. 

*b. She could have an infection. 

c . She needs to see a specialist. 

d. She should go to the hospital. 

Prototype item notes 

The aural stimulus is a long institutional dialogue between a doctor and 
a child^s mother. The test taker must interpret several phrases in the 
text to find the correct answer. Option A incorporates the word 
"antibiotics” heard in the text but demonstrates a misunderstanding of 
the information presented. Options C and D plausibly relate to the 
context but are unsupported by the text. 

Note: Prototype DM-L2 was modified based on pilot testing results and 
thus the estimated range and discrimination point are not available. 
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Listening item specifications 



Draw meaning 



DML Notes 



1. Item type DMLl was pilot tested with the select-two-correct format. Pilot 
testing results indicated that drawing two meanings was problematic, 
particularly for test takers at the lower levels, and the item type was 
therefore modified to the multiple-choice format. Pretesting of this item 
type also emphasized the importance of pictures that clearly represent 
the intended meaning. One way to control for clarity is to have a small 
number of test takers at the pretesting stage describe aurally what they 
see in the pictures. Where there are discrepancies, modifications 
should be made. 

2. Additional item types that assess test taker ability to draw meaning 
from a listening text need to be developed. Listening texts that lend 
themselves to assessing this skill need to be selected and item types 
developed following the prototyping process described in Kahn et al. 
(1995). Item types that use a constructed-response format should be 
explored. 



Listening item specifications 



Extract global information 



'Eatract Global Information (EG) 



General description 

The test taker will listen to an extended monologue or dialogue and 
answer a general question requiring the interpretation of ideas 
presented in several utterances throughout the text. 

Text type 

Extended monologue or dialogue 
Target level 

Intermediate low through advanced low 

Test level 

Level A: essential 
Level B: essential 

Item types 

In item type EGl, the test taker numbers a series of topics in the order 
they are discussed. In item type EG2, the test taker selects two of five 
responses to a factual question about a text. 
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Listening item specifications 



Extract global information 




Item Type EGl 



Item format 

Sequencing 

Test level 

Appropriate for both Level A and Level B tests 
Stimulus attributes 

The aural stimulus is an extended monologue or dialogue containing 
three distinct topics. The test taker listens to the text once in its entirety 
to answer the question. 

On Level A tests, and optionally on Level B tests, there is a visual 
stimulus that orients the test taker to the content of the text. 

The item stem consists of the following instruction: 

Number the topics (1, 2, 3) in the order they are discussed. 

A set of three words or phrases follows the instruction. Each word or 
phrase describes one of the topics discussed in the text and, where 
possible, uses words other than those found in the aural stimulus. The 
topics are presented in random order. 

The aural stimulus contains target-level language. The words or 
phrases in the item are at a lower level. 

Response attributes 

Each word or phrase is preceded by a blank, in which the test taker 
must write a nvunber corresponding to the ordering of topics in the text. 
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Listening item specifications 



Extract global information 



Prototype item 



Aural stimulus 



You will hear a conversation that takes place in a doctor^s office. The doctor 
will discuss three general topics with Natalie's mother. Read the topics now. 
(Pause five seconds.) Now, listen to the conversation and number the topics in 
the order they are discussed. 



Doctor: 

Mother: 

Doctor: 

Mother: 

Doctor: 

Mother: 

Doctor: 

Mother: 

Doctor: 

Mother: 

Doctor: 

Mother: 

Doctor: 

Mother: 

Doctor: 

Mother: 



Good morning, Fm Dr. Li. 

Hi Dr. Li, Fm Mrs. Johnson. 

So, what brings you here today"? 

Well, my daughter hasn't been feeling well and Fm a little concerned. 
Okay. Well, what's your daughter's name? 

Natalie. 

And how old is she? 

She's five years old. 

Okay. And what are her symptoms? 

She has a sore throat and a fever. 

How long has she been sick? 

Well, let's see. Her sore throat started a few days ago and last night 
when I took her temperature she had a slight fever. 

All right. Has she been eating normally? 

More or less, but it's been a little hard for her to eat. Her throat is really 
sore and it hurts when she swallows. 

How has she been acting? 

A little tired. She's been sleeping more than she usually does. 



Doctor: 

Mother: 

Doctor: 

Mother: 

Doctor: 

Mother: 

Doctor: 

Mother: 

Doctor: 

Mother: 



Okay. What have you been doing for Natalie? 

Well, we kept her home from school yesterday and made sure she got 
plenty of rest. 

Anything else? 

We gave her a lot of juice and water to drink. 

Have you given her any medicine? 

No, not yet. 

Okay. Are any of Natalie's friends at school sick ? 

Not that I know of. 

Is anyone else in the family sick? 

Well, my son was home sick last week and I just got over a cold a 
few days ago. 



Doctor: I see. Is she generally a healthy child? 

Mother: Yes. 

Doctor: Okay. Is she allergic to any medicines? 

Mother: Not that I know of. 

Doctor: Has she had any surgeries in the past? 

Mother: No. 

Doctor: Okay. Any medical problems in your family that I should be aware of? 

Mother: No. 

Doctor: Okay. Fm going to examine Natalie now. 
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Listening item specifications 



Extract global information 



Test booklet 



Number the topics (1, 2, 3) in the order they are discussed. 



Natalie’s S5onptoms 

Natalie’s medical history 

What Natalie’s parents have done 



Prototype item notes 

The aural stimulus is an extended institutional dialogue between a 
doctor and a child’s mother in which three general topics are discussed. 
The test taker must listen to the entire dialogue and order the topics in 
the order they are discussed. 

Estimated range: IH-AH Discrimination point: AL/AH 
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Listening item specifications 



Extract global information 



Item Type EG2 



Item format 

Select two correct 

Test level 

Appropriate for both Level A and Level B tests 
Stimulus attrihutes 

The aural stimulus is an extended monologue or dialogue describing a 
single person, phenomenon, or event. The test taker listens to the text 
once in its entirety to answer the question. 

On Level A tests, and optionally on Level B tests, there is a visual 
stimulus that orients the test taker to the content of the text. 

The item stem consists of a general question asking for important ideas 
repeated or referred to in several places in the text. 

The aural stimulus contains target-level language. The words or 
phrases in the item are at a lower level. 

Response attributes 

A set of five single-word or single-phrase options follows each stem. The 
response options use words or phrases different from those found in the 
aural stimulus, where possible, and contain language at a lower level 
than the target level. The test taker must select the two correct answers. 
The keys are words or phrases answering the question using words 
similar to those in the aural stimulus. 

The three distractors are grammatically accurate and incorporate 
words or phrases from the text. The distractors plausibly relate to the 
content of the text but are inaccurate in that they are either unsupported 
or .contradicted by information in the text. 
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Listening item specifications 



Extract global information 



Prototype item 
Aural stimulus 

You will hear a news report about Philip Vera Cruz. Look at the question. 
(Pause five seconds.) Now, listen to the news report and check the two 
correct answers. 

Philip Vera Cruz passed away in Bakersfield, California yesterday. He 
was 89 years old. Vera Cruz was a migrant worker who became a political 
activist and labor organizer in California. He is best remembered as an 
organizer of Filipino workers and co-founder of the United Farm Workers 
Union. 

Vera Cruz immigrated from the Philippines when he was 22 years old, 
hoping to get a good job and an education. However, like many other 
Filipino immigrants who came to the United States in the 1920s, he was only 
able to find low-paying jobs in factories, canneries, and fields. Because 
these jobs were short-term, Vera Cruz had to move frequently in order to 
find work: cutting boxes in a factory in Washington, canning fish in 
Alaska, and picking grapes in California. 



Vera Cruz was paid very little, yet he sent most of the money he earned to his 
family in the Philippines. He never married, fearing that if he had his own 
family to take care of in the States, he would not be able to help his family in 
the Philippines. 

Vera Cruz attended college briefly, but his limited income and commitment 
to supporting his family made it impossible for him to realize his dream of 
becoming a lawyer. He did, however, succeed in making education 
possible for his family: he helped his sister, his brother, and all his brother's 
children go to college. 

Vera Cruz was angered by the low wages, long hours, and poor living 
standards of migrant workers, and in the mid-1960s helped organize the 
Filipino workers to fight for improved working conditions. In 1965 he 
joined forces with Cesar Chavez to establish the United Farm Workers 
Union, where he served as vice president until 1977. Because of his deep 
concern for workers and anger at the way they were treated, Vera Cruz 
dedicated his life to fighting for social justice and equality. He will be 
remembered as a great leader and activist. 
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Listening item specifications 



Extract global information 



Test booklet 



Check (V) the two (2) correct answers. 



Which two words or phrases best describe Philip Vera Cruz? 

lawyer 

educator 

farm owner 

migrant worker* 

union organizer* 



Prototype item notes 

The aural stimulus, an extended monologue, is a news report about a 
famous person. The test taker must listen to the entire monologue to 
find the two words or phrases that best describe the person referred to in 
the text. The three distractors incorporate words or phrases heard in 
the text but are contradicted by information presented in the text. 

Estimated range: IH-AH Discrimination point: AL/AH 
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Listening item specifications 



Extract global information 



EG Notes 



1. Item t5T)e EGl and item type EG2 were found to be promising not only 
because they discriminated well but also because they helped orient the 
test taker to the listening text. However, different listening texts may 
lend themselves to additional item types other than those specified here. 
It is recommended that these item types be developed for both Level A 
and Level B tests. 
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Listening item specifications 



Make inferences 






Make Inferences (MIL) 



General description 

The test taker will draw conclusions about ideas implied but not 
explicitly stated in a listening text. 

Text type 

Medium or long monologue or dialogue 
Target level 

Intermediate high through advanced high 

Test level 

Level A: optional 
Level B: essential 

Item types 

In item type MILl, the test taker selects one of the four responses to a 
question about a text. 
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Listening item specifications 



Make inferences 



Item Type MILl 



Item format 

Multiple choice 

Test level 

Appropriate for Level B tests only 
Stimulus attributes 

The aural stimulus is a medium, long, or extended monologue or 
dialogue or any combination of the three. The text should contain at 
least one idea that is not explicitly stated and several contextual clues 
that allow the listener to draw only one conclusion about the unstated 
information. A long text is heard two times. A segment is heard one 
time after the extended text is heard in its entirety. 

The item stem consists of a question asking for information implied but 
not explicitly stated in the text. 

The aural stimulus contains target-level language. The item stem 
contains language at a lower level. 

Response attributes 

A set of four single-phrase or single-sentence options follows each stem. 
The response options contain language at a lower level than the target 
level. The test taker must circle the letter of the correct answer. The key 
is a phrase or sentence answering the question using words other than 
those found in the text. 

The three distractors are grammatically accurate and plausibly relate to 
the content of the text but are inaccurate in that they are either 
unsupported or contradicted by information in the text. At least one of 
the distractors incorporates words or phrases from the text. 



Listening item specifications 



Make inferences 



) 




Prototype item 



Aural stimulus 

Now you will hear another conversation between Dr. Li and Natalie's mother. 
You will hear the conversation two times. Listen to the conversation and circle 
the letter of the correct answer. 



Doctor: 

Mother: 

Doctor: 

Mother: 

Doctor: 

Mother: 

Doctor: 



Mother: 

Doctor: 

Mother: 

Doctor: 

Mother: 

Doctor: 

Mother: 



Everything seems fine, but her throat's a little red. I want to take a 
throat culture to make sure she doesn't have an infection. 

Okay. So, um, how long will it take for the test results to come back? 
We should have the results in two days. You should call us then to see 
if they're ready. 

All right. So, she might have a throat infection. You're going to do a 
test and I should call you back in two days to see what the results are. 
Uh-huh. And if she does have a throat infection I'll prescribe some 
antibiotics at that time. 

Okay. What should I do in the meantime? 

You can get some children's Tylenol at the pharmacy. Give it to her 
every four to six hours until her fever has gone down. She should 
continue to drink a lot of liquids and get plenty of rest. 

Okay. How long should I keep her out of school? 

It's best to keep her home for the next few days. In fact. I'd like to see 
her again before you send her back to school. 

Okay. So should I schedule a follow-up appointment? 

Yes, you can do that on your way out. 

Fine. By the way, is it all right for her to be around her brother? 

Yes, as long as she washes her hands frequently and they don't share 
utensils or eat off the same plate. 

All right. That sounds good. Thank you very much Doctor Li. 



Test booklet 



Circle the letter of the correct answer. 

When does this conversation take place? 

a . one week later 

b. a few days later 

*c. after the examination 
d. after Dr. Li sees the test results 



Prototype item notes 

The aural stimulus is a long institutional dialogue between a doctor and 
a child’s mother that follows an extended dialogue between the same 
speakers (see p. 148). The test taker must infer from contextual clues in 
both texts when the conversation takes place. The three distractors 
incorporate words and phrases heard in the text and represent plausible 
responses to the question but are contradicted by information presented 
in the text. 

Estimated range: AL-AH Discrimination point: AH+ 
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Listening item specifications 



Make inferences 



MIL Notes 



1. Additional item types that assess test taker ability to make inferences 
need to be developed. Listening texts that lend themselves to assessing 
this skill need to be selected and item types developed following the 
prototyping process described in Kahn et al. U995). 



Glossary of selected terms 



Glossary of Selected Terms 



The terms in this glossary are defined to reflect how they are used in this 
test development plan and cannot necessarily be generalized beyond this 
context. 

constructed response 

An item format requiring the test taker to produce rather than select a 
response 

discrimination point 

The point at which an item best differentiates performance between two 
adjacent proficiency levels 

distractor 

An incorrect response option 
essential skills 

Skills identified by working group members as critical for placement 
and mandatory on every form of the test 

estimated range 

The range of proficiency levels at which an item seems most appropriate 
based on content considerations and statistical performance 

factual function 

One of three categories of functions referred to in the Model Standards 
involving eliciting or providing information about facts. Examples of 
factual tasks include identify, tell, describe, explain, and analyze. 

fill in the blank 

An item format that requires the test taker to provide pieces of missing 
information consisting of a single number, word, or phrase in the 
spaces provided 

function 

The communicative purpose of a task. Three basic categories of 
functions referred to in the Model Standards are: factual, social, and 
suasive. (See definitions of specific functions in this glossary.) 



Glossary of selected terms 



institutional 

Institutional refers to situations in which the participants involved 
interact in clearly defined, routine roles generally characterized by 
formal register. Examples include doctor and patient, clerk and 
customer, and teacher and student interactions. 

item stem 

A question or statement contained in the stimulus to which the test 
taker responds 

key 

The correct response(s) 

Level A test 

An instrument with content appropriate for Model Standards 
proficiency levels beginning low through intermediate high 

Level B test 

An instrument with content appropriate for Model Standards 
proficiency levels intermediate low through advanced high 

matching 

An item format that requires the test taker to identify associated 
elements (picture, word, phrase, or sentence) from two parallel groups 

multiple choice 

An item format that requires the test taker to select a single correct 
option from several choices 

optional skills 

Skills identified by working group members as important but not critical 
for placement and which need not be on every form of the test 

passes 

The number of times the test taker hears a listening text 



niJnf. tpstixtff 

M. C7 

Administration of items or whole tests to a large number of students 
across an appropriate range of levels 
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Glossary of selected terms 



pretesting 

Administration of texts and associated items to a small number of 
students at targeted levels 

primary text 

The main text on which most reading items and writing tasks are 
based. A primary text is either a display, paragraph, or multi- 
paragraph passage. 

prompt 

A series of sentences that specify the elements of a writing task 



prototype 

A model item or text that serves as an example for item writers 



response attributes 

Characteristics of the response for a given item including descriptions 
of correct and incorrect response options (selected response) and 
guidelines for judging the acceptability of the test taker’s response 
(constructed response) 



secondary text 

A supplementary text related to the primary text and critical to some 
reading and writing task types. A secondary text is either an additional 
display or paragraph or a statement in the item stem which sets up a 
condition or problem. 

select [number] correct 

An item format that requires the test taker to select more than one 
correct answer from several response options. The number of correct 
answers is always specified. 

selected response 

An item format requiring the test taker to choose the correct answer(s) 
from a series of response options 



sequencing 

An item format that reqmres the test taker to order a series of events or 
topics 
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Glossary of selected terms 



social function 

One of three categories of functions referred to in the Model Standards 
involving eliciting or providing interpersonal information in order to 
fulfill a social purpose. Examples of social tasks include introduce, 
show gratitude, apologize, and express values. 

stimulus attributes 

Characteristics of the input to the test taker for a given item including 
descriptions of the text(s), item stem, and visuals, where appropriate 

suasive function 

One of three categories of functions referred to in the Model Standards 
involving eliciting or providing information in such a way that a 
persuasive purpose is fulfilled. Examples of suasive tasks include 
request, suggest, advise, and recommend. 

target-level language 

Language presumed to be familiar to test takers at a given proficiency 
level, but unfamiliar to test takers at the next lower proficiency level 

turn 

The uninterrupted speech of a single person 
unique answer 

An item format that requires the test taker to generate a short response 
consisting of a single number, word, or phrase to a question that has 
only one correct answer 
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Mark Larroca^ 
Rachel Khattab® 


City College of San Francisco 


Nadia Scholnick 


Fremont School for Adults, Sacramento 


Mary White 


Hayward Adult School 


Joyce Clapp 


Los Angeles Unified School District 

Division of Adult and Career Education 


Barbara Martinez 
Greta Kojima® 


Merced Adult School 


Debbie Glass 


Mt. Diablo Adult Education 


Jacques LaCour 


Oxnard Adult School 


Judy Hanlon 


San Diego Community College District 
Continuing Education Centers 


Gretchen Bitterlin^ 
Leann Howard‘S 


Santa Clara Unified School District 
Educational Options 


Bet Messmer 


Torrance Adult School 


Bertie Wood 


Watsonville Adult School 


Claudia Grossi 




^Served as representative during Year 1 

^Served as representative during Year 2 

"^Served as a representative during Years 1 and 2 

^For this agency two representatives shared responsibility. 
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Appendix B 



Criteria for Estimating the Range and Discrimination Point 
of Prototype Reading and Listening Items 
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Criteria for Estimating the Range and Discrimination Point 
Prototype Reading and Listening Items 

The appropriateness of items at different proficiency levels is based 
^th on content considerations and on item performance at those levels. 
1^0 indicators ^ item appropriateness are given for each prototype item: 
the Estimated Range (the range of levels at which the item provides 
appropnate discrimination) and the Discrimination Point (the two adjacent 
levels between which the item best discriminates). These indicators were 
denved through a comparison of the item content (including the skill being 

1 ® language in the item and the primary text) with the 

odel Standards, and through the application of operational criteria from 
pilot testing, described below. 

In the majority of cases, the empirical performance of the items was 
consistent with the judgments based on item content. In a few cases 
however, discrimination between two levels failed to meet the operational’ 
criteria where item content predicted that the item would discriminate 
between those levels. In these cases, content considerations overrode the 
empirical data, primarily because the Model Standards levels of students at 
agencies where pilot testing took place had not been precisely determined 
and thus it could not be said with certainty that the item would not 
discnminate between those levels. 

The operational criteria for determining the Estimated Range and 
Discrimination Point of items are as follows: 

Estimated Range 

If the p value of the item at all levels tested (BL-IH on Level A or IL-AH 

on Level B) exceeds .75, the item is considered too easy for the 
population. ^ 

If the p value of the item at all levels tested (BL-IH on Level A or IL-AH 

on Level B) is less than .35, the item is considered too difficult for the 
population. 

An itein is defined as discriminating between two adjacent levels if the p 
value at the higher level is at least .10 greater than the p value at the 
ower level unless (a thep value at both the higher and lower level is 

less than .35 or (b) the p value at both the higher and lower level is 
greater than .75. 

If an item administered to four adjacent levels (e.g., BL-IH) 
discriminates between the two lowest levels (e.g., BL-BH) and the two 
highest levels (e.g., IL-IH), it is defined as discriminating between the 
t\^ middle levels (e.g., BH-IL) regardless of the magnitude of the 
diiierenc© in p values between those two levels. 



Discrimination Point 

The discrimination point indicates the two adjacent levels between 
which an item best discriminates. It is operationally defined as the 
point at which the p value at the lower level is less than .5 and the p 
value at the higher level is greater than .5, as long as the difference 
between these two p values is greater than .10. If not, the 
discrimination point of the item is the next higher or lower point, 
whichever has the greater difference between the two p values. If no 
point has a difference in p values greater than .10, the item is 
considered to have poor discrimination and should not be used. 

Figure 7 below summarizes the discrimination points for Level A and 
Level B referred to in this document. 

Figure 7 

Definitions of Discrimination Points 



Test 

Level 


Discrimination 

Point 


Levels between which an item best discriminates 


A 


BL- 


Beginning low and below (beginning literacy) 


BL/BH 


Beginning low and beginning high 


BH/IL 


Beginning high and intermediate low 


IL/IH 


Intermediate low and intermediate high 


IH+ 


Intermediate high and above 


B 


IL- 


Intermediate low and below 


IL/IH 


Intermediate low and intermediate high 


IH/AL 


Intermediate high and advanced low 


AL/AH 


Advanced low and advanced high 


AH+ 


Advanced high and above 
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Appendix C 

Examples of Reading/Writing Text Adaptation 



BeadinglWriting Text Adaptation 



This appendix illustrates the guidelines for text adaptation presented 
m the Reading/Writing Text Specifications above by providing examples of 
texts in their ori^nal and adapted forms. Although all texts underwent 
minor modifications such as changes in proper names, telephone 
numbers, and addresses, the texts presented here underwent extensive 
revision. Adaptations made on the three texts presented below (a bicycle 
shop advertisement, an apartment guide, and a newspaper article) are 
representative of the kinds of changes necessary for a text to be appropriate 
for placement testing. The original and adapted versions of each text are 
accompanied by explanatory notes highlighting the revisions made. 



■ •,> < 










175 



164 



Bicycle Shop Advertisement 



Original text 




'Donnelly Yellow Pages, Los Angeles, 1992 



best copy available 




1G5 
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Adapted text 



Village Bike Shop 



WE SHIP 



World Class Bicycles 
Call: 257-2453 



I VISA I 




WORLDWIDE 

854 Ocean Blvd • Palm Village 

• Bicycles for the entire family 

• Free lifetime service plan with purchase of any bike 

• All makes & models expertly serviced 

• Quality selection of accessories & apparel 

• Bicycle & exerciser rentals available 



adapted from Donnelly Yellow Pages, Los Angeles, 1992 



The bicycle shop advertisement was simplified to focus on 
distinguishing features and to make the presentation less cluttered. 
Some features such as specific bicycle types and brand names were 



The layout was edited for visual clarity. Critical information was listed 
in one area or made more prominent, redundant information was 
deleted, and visuals were made simpler and smaller. 



Free Parking Available 
★ OPEN 7 DAYS ★ 




Adaptation notes 



deleted. 



Apartment Guide 



Original text 



PROPERTY REVIEW <jUIDE^^ 



CsCall for details 
5>i5in9le j 

Oty name 


Phone 


4 


A 


Special 






1 


P 




1 






I 


/ rcATune 


1 COASTAL CITIES 
























z 










SANTA MONICA 




































Pacific Ocean Apts. 


310 399-8583 


17 


L5 


Call manager 


S.1 


1 


♦ 


♦ 


♦ 














c 


Ctoae to beach 


Pico Lanai 


310 B2B-9389 


16 


Q9 


Low rents 


1 


1 


♦ 




♦ 




♦ 












Great valuel 


MARMADB.REY 




































Adnvalty 


310 821-4282 


2T 


W12 


2 months Free 


1.2 


1.2 










"T 












Lap Pool 


BarHaitxy 


310 823-4689 


19 


W12 


1 month Free 


S.1.2 


1.2 




♦ 


16 




1 


T~ 


T 










Casa de Manna 


310 822-8700 


22 


W12 


Move>in SpedsJ 


1.2 


1 




♦ 


4 




2 


2 


♦ 






c 


rteBtoPMe 


Deauville Maiina 


310 823-4655 


19 


W12 


11/2 month Free 


1.2,3 


1.2 


♦ 


♦ 


4^ 




1 


1 








c 


BoataiK 


Islander 


i 310 328-3747 


21 


W12 


1 Month Free 


1.2 


1.2 


♦ 


♦ 

~ 


4 




1 


1 


T 








BoatSps 


Marina City Club 


i 310 305-9191 


23 


W12 




1.2,3 


2 


♦ 




♦ 




♦ 


♦ 


"i" 




♦ 


c 


PerthouBM 


Marina Harbor 


I 310 823-8974 


3 


W12 


1 month Free 


1.2,3 


1.2 










5 


3 


♦ 


"5" 






BoMStoa 


Marina Pointe 


310 822-6807 


85 


W12 


rritTimenffiflnti 


1.2 


1.2 






♦ 




♦ 


♦ 


♦ 






1C 


Qrardopering 


Marina Towers 


310 822-9091 


20 


W12 


Move-In bonus 


S.1 


1 






2 














n 


Ocean View 



The Southern California Apartment Magazine, April 20-May 3, 1994 



BEST COPY AVAILABLE 
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Adapted text 



Apartment Guide 



Apt. 

Features 


Willow 

Court 


Village 

Place 


Hamill 

Towers 


Pierce 

Plaza 


rent 


$750 


$550 


$700 


$950 


bdrms. 


2 


1 


2 


2 


baths 


2 


1 


1 


2 


parking 




♦ 


♦ 


♦ 


security bldg. 


♦ 




♦ 


♦ 


pool 


♦ 


♦ 






laundry 


♦ 




♦ 


♦ 


pets ok 




♦ 


♦ 




unique feature(s) 


* on bus line 


• all new 
appliances 


• all utilities 
paid 


• A/C 

• cable ready 



adapted from The Southern California Apartment Magazine, April 20-May 3, 1994 




Adaptation notes 



The apartment guide was simplified to focus on a restricted number of 
distinguishing features and rental prices, an essential feature not 
present in the original version, were added. Unique features listed in 
the original version were replaced with features considered more 
appropriate for the target population. Other modifications included the 
addition of abbreviations (e.g., “bdrms.”) and the substitution of jargon 
with more common terminology (e.g., “security bldg.” for “control 
access”). 

The layout was modified for visual clarity. Only one code was used to 
indicate presence or absence of a given feature, fewer apartments were 
presented, and features were listed horizontally rather than diagonally. 
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Newspaper Article 



Original text 



Graduation No Minor Task For Adult Student 



By April Hawarden 



After spending six years studying 
at the Culver City Adult School, 
Minor Rodriguez’ primary 
motivation for graduating was his 
18>year>old stepdaughter, who 
becomes a high school senior this 
fall. 

“[!) wanted to have my diploma 
first,” Rodriguez explained. 

Rodriguez, who attended classes 
at the adult school to learn English 
and become a U.S. citizen, did not 
expect to accomplish two feats in 
one year: getting his diploma from 
the adult school an being named 
Citizen of the Year by California 
State Consortium for Adult 
Education. 

The consortium, comprised of 
California adult schools, attempts to 
raise awareness of adult school 
programs and course offerings. 

Described as modest by one 
teacher, Rodriguez did not expect 
the recognition. 

“1 feel proud,” the 33-year-old 
student said. 

But Rodriguez has many reasons 
for graduating. “[I] wanted to do 
this as an example for my kids,” 
citing the fact that he came from 
another country, Guatemala, and 
did not know the language, but still 
received an education. 

Rodriguez also has two 
daughters ages 12, 9 and a S-year- 
old son. 

“It’s up to them,” he said. “1 
have a lot of confidence they will be 
better.” 

When Rodriguez first came to 
the United States IS years ago, he 
felt no immediate need to learn 
English since the people he hung 
around spoke primarily Spanish and 
he listened to Spanish radio and 
TV stations. 



Raised in single parent home by 
his mother because his father died 
when he was young, Rodriguez said 
his family “taught me to be 
responsible, to be straight.” 

Since he wanted something 
better out of life, Rodriguez went 
back to school in 1988 to learn 
English and become a citizen. 

He said one word describes his 
decision to go to school, raise a family 
and hold two jobs — sacrifice. 

But with all he has done, 
Rodriguez said he feels good about 
his accomplishments. “I can share it 
with my kids.” 

In order to support his family, 
Rodriguez works as a storeroom 
worker at Kaiser Permanente Medical 
Care Program in West Los Angeles 
during the week and as a nurses’ 
aide at Huntington Memorial 
Hospital in Pasadena on the 
weekends. 

Many teachers helped Rodriguez 
throughout the years, but the most 
influential teacher was Harriet 
Degelsmith. 

“She always makes me feel good 
about myself’ and she was “always 
finding ways to make it easier,” 
Rodriguez said. 

In response to this remark, 
Degelsmith said “I guess that’ s my 
personality, but I’m also paid to do 
that.” 

Degelsmith described Rodriguez 
as “very diligent and very focused,” 
which is necessary when a person 
has a family, two jobs and 
participates in community activities. 

In regard to his award, 
Degelsmith said, “We are very proud 
... we are proud to think we had a 
part in that.” 

She said the adult school is full of 
students like Rodriguez, who have 



many responsibilities, but Rodriguez 
stands out because he gets the job 
done with a smile and in a timely 
fashion. 

She said he is “one of those 
people you would feel privileged to 
know” and added that Rodriguez 
would be embarrassed by her 
remarks because he is very modest. 

He’s so modest in fact that 
Rodriguez did not mention receiving 
an employee of the year award from 
Kaiser. 

Rodriguez said he was named 
employee of the second quarter and 
Most Outstanding Employee for 
1990 in Kaiser’s material 
management department. 

His supervisor, Lloyd Daron, said 
he is impressed by Rodriguez’ 
stamina. 

“That’s commendable he had 
the fortitude to do that,” he said, 
and added that Rodriguez has also 
been in charge of the blood drive 
and worked diligently at getting 
donors. “He’s quite a fella.” 

Culver City Adult School 
Counselor Nicki McBreen said there 
were other students considered in 
the selection process, but Rodriguez 
“was sort of at the top” because he 
had taken more classes than the 
other candidates and remained 
dedicated. 

“Minor has slowly but surely 
stuck with the program,” McBreen 
said. “He’s really a great guy.” 

Receiving his diploma is only the 
first step to other ambitions for 
Rodriguez. He said he wants to go to 
college and earn a degree in 
radiology. 



Culver City News, June 23, 1994 



Adapted text 



City News 




City News* Person of the 
Week is adult school student 
Oscar Rodriguez, whose dream 
came true last week when he 
received his high school 
diploma after six years of study 
at City Adult School. 

Rodriguez said his primary 
motivation for graduating was 
his 17-year-old daughter, who 
becomes a high school senior 
this fall. “I wanted to have my 
diploma first,’* the 35-year-old 
student explained. 

Rodriguez, who attended 
classes at the adult school to 



learn English and become a U.S. 
citizen, accomplished two feats 
in one year: getting his high 
school diploma and being named 
Citizen of the Year by the State 
Consortium for Adult Education. 

Rodriguez came to the United 
States 14 years ago from 
Guatemala. At that time, he felt 
no immediate need to learn 
English since the people he 
spent time with spoke primarily 
Spanish and he listened to 
Spanish radio and TV. 

Later, he realized he needed to 
learn English in order to get his 
high school diploma. So, in 
1988 he enrolled in City Adult 
School where he proved to be a 
very diligent and focused 
student. He found time to study 
and participate in community 
activities while working two 
jobs and raising a family. 

According to Consortium 
president Gerry Delaney, “The 
adult school is full of students 
like Rodriguez who have many 
responsibilities, but Rodriguez 



Thursday, June 23, 1994 

stands out because he always 
gets the job done with a smile and 
in a timely fashion.** The 
Consortium acknowledged 
Rodriguez* accomplishments by 
giving him the Citizen of the Year 
award. 

“I feel proud. I wanted to do 
this as an example for my kids,** 
said Rodriguez, who has another 
daughter, age 7, and two sons, 15 
and 10. 

In fact, Rodriguez has set a 
good example not only at school 
but at work. Last year he was 
named Most Outstanding 
Employee at County Hospital 
where he works as a nurses* aide. 

Receiving his diploma is only 
the first step to other ambitions 
for Rodriguez, who plans to 
continue his work in the medical 
field. “Now I want to go to 
college so I can become a nurse.** 

City News congratulates Oscar 
Rodriguez and wishes him 
continued success. 



adapted from Culver City News. June 23, 1994 



Adaptation notes 

The newspaper article was edited to develop specific ideas presented in 
the original text and to clearly delineate the reasons Rodriguez was 
chosen as person of the week. Such revisions were necessary in order to 
assess the ability of the test taker to interpret relationships between 
ideas. Although the newspaper article underwent extensive revision, 
the journalistic register was retained and vocabulary and structures of 
the same complexity found in the original were used. 
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Appendix D 



Sample Responses for Generate Writing Tasks 



Sample Responses for Generate Writing Tasks 



This appendix contains sample responses for Generate writing 
tasks. There is one writing sample per level for each prototype task. The 
sample responses are accompanied by notes explaining the application of 
the scoring rubric to the sample with regard to task fulfillment and 
language use. Also included in this appendix are examples from test 
takers who should be retested with a higher or lower level writing task. 
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Prototype Task GNl 



Beginning low (1) 




None of the task requirements are fulfilled; unrelated words and 
phrases form less than a single sentence 



Prototype Task GNl 



Beginning high (2) 



Just a note . . . 



Mypyjr-i/1 — tne — — 9-fi 

/o .I nL fA-f)ri hnJ 



r rYisnCtr> r\i IV? Q I — VlflV 

prcjaiz-^vT) 



Some task requirements are fulfilled; one or possibly two simple 
sentences with frequent errors 



O 

ERIC 
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Prototype Task GNl 



Intermediate low (3) 




All task requirements are fulfilled; simple sentences form a short 
paragraph; errors are frequent 



BEST COPY AVAILABLE 
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Prototype Task GNl 



Intermediate high (4) 



Just a note . . . 



Vi 



Cy ^ tc ^CLUJ 



CMr Tin 






\rA loW X — PpcMg — .^rPirL- 

utvY a, fyrte. Lfjny^ q/?~Vn 



W\f 



VV<=>, r/oHcgy^ jm_ ■iuU 16 H- gjf/ji ypT^ 

"X 'yy) sn Ji/// tj± d L 






'XT uji // c/fSf '/ yt/fCf — 



A/g^yjg, L// 'T^ i/JOA'/ qo- 



I/I 

^ 



ejanBf/7 -rna^ tuhi r/a 



/^iot/'/ iJfj/f T^ 'yf/ ? p //'05C 



Unact/ // yu Utz/J /71/ 




74/L 



All task requirements are fulfilled; related sentences form a sequential 
paragraph; some complex structures attempted; errors in simple 
structures 



f. 






••■ S,” 



> >V < 
n-i rfr 
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Prototype Task GN2: Level A 



Beginning low (1) 



Dear Chris, 


You won’t believe what happened to me the other dav! 


dLOOi>»<7 Ml ru - V 


4a e y 






































None of the task requirements are fulfilled; unrelated words and 
phrases form less than a single sentence 
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Prototype Task GN2: Level A 



Beginning high (2) 





None of the task requirements are fulfilled; one sentence with frequent 
errors 



er|c 
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Prototype Task GN2: Level A 



Intermediate low (3) 



Dear Chris, 


You won’t believe what haooened to me the other dav! 


OUA CettMe SLfi 


u)Lp\a t umrk \ralp hCtfC(l\L. 




iJjLsiA ytau s^au/ Ate JAa 




. lUfi m/yiA — U& — — 


iytiAk Al,ir> iMAuJ-ii ^pA . A^d "% 


La A Ia -Ia JLa — auA. 

























Some of the task requirements are fulfilled; sentences form a short 
paragraph; errors are frequent 
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Prototype Task GN2: Level A 



Intermediate high (4) 



Dear Chris, 


You won’t believe what haoDened to me the other day! 




1 ” 


jbUM\AJL . 


^(/vaam>v^ fbict^ruiuL, A-tftX . <59^ '^>r\.duK 




uaA> 0 /yyu/vut 


cajJlxi^ , T^Ux. <Y^ -Awi/uit 




J^giXcuiaMtii. /(At OA^XAiiaL ^ 


pj^Uut . 6je^ cOAiifyJ^ uxJiLt’ . 


|«AA<ntc !{ 



















All task requirements are fulfilled; related sentences form a sequential 
paragraph; complex structures attempted; some errors in simple 
structures 
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Prototype Task GN2: Level A 



Retest Level B (5) 



Dear Chris, 


You won’t believe what hapoened to me the other davi 


T t/j At* \i\afiu L 1 — 


^ t ^ * 


iii \k\eA6H\ naiX rnm fiiV.-l aA^ie<i vnt. tfa. 


XffniO- Ahi ireXA 


9irey> Ue "L \)JaK WAA rltU 


hte^AJ/L Ir\-fuj>. crtirmiliA e/ijJxtA — = — 


■ftP+lrAuaCj 1 fjdb^ koWju Irt Wi Z kaAhdhA 


" ■ 5 j ^ r 


T uJ/iA. iLt ac>lie2 , 4^ ii^jx2iA 


%tUL ajUCLlXk ijoi' iut. ijitt kih — \uaiJijt 






* d . ® 

T x4-Bji 


1 lifuX JHhieJiJ 




^ — — _ 


PS s Aom.V Ln qft -to OJH^^WeX <mx. 


0 ^ 





All task requirements are fulfilled; complex sentences usually 
attempted; some errors in complex structures 

Note: The test taker fulfills all task requirements and demonstrates a 
command of the language that seems to exceed capabilities 
representative of intermediate high. Therefore, retesting at the higher 
level is recommended. 

BEST COPY AVAIMBlbE 
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Prototype Task GN2: Level B 



Intermediate low (3) 



Dear Chris, 


You won't believe what haooened to me the other dav! 


N'vw/ lire: vise- 


-l-WyolC . ~TvA./r> lvnoi^W*\ 


4-kf<5 vvlr\M 4-vf<P-cl Vtdo r fv^ 


iplnr^ T XKinv'Kcr . or>-l- tr\ nr»rl 


<:hn^KlQ:cl -Uio \\Y\c^r(^c- 


nmA T YTpryA vsA\! \'\r\W\n- nn 4-l\^ \ i nr(>r — 





























Some requirements of the task are fulfilled; ideas are loosely integrated 
and are not developed; simple sentences with frequent errors 



Prototype Task GN2: Level B 



Intermediate high (4) 



Dear Chris, 


You won’t believe what happened to me the other day! 






ywe -Itwg. to tell it4 for 


yoii . X 'W'y 


/ f 1 

W a ?oy 


\xJ\! heftir At c-foire. 


A Cfol^lA , 


r^n YnM iVfl^fne^ K/m; T U//14 


1 ^ 

(vlie.M X ^a\Al Y0\i 


1 noftV^ 


i-t . Co X rrt(le/i V/ d4\«^ 


^ofi'^ce, 'Tke, i?oj/ f ^ 


he/iv^J tke pcii'c^ . 


v»^M Aivo.y -{Vdvw Cto^r^. 


r rflU^ lo^liV,i/e. f*f7 vietjji-iA wy 






<,tr\CeveW VoU^^. 


[:?p«qlA.< n^Aver 



Most requirements of the task are fulfilled; ideas are somewhat 
integrated and developed; mostly simple sentences with some complex 
structures, advanced vocabulary, and complex verb tenses 
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Prototype Task GN2: Level B 



Advanced low (5) 



Dear Chris, 


You won't believe what happened to me the other dav! Po 


\//ni vviv hick war Hv 


vnv dvivor'j 


0^0 ? X ciof iV l7oct ^Iyi9<Iv Wcai/JC ^rc wai 


^ on-jirv wKcs-t 'iwne. >5 Msikc^n d^i'ru 


•h> buy ^t-V "(vTvyi ■vrrt- . X sflW h>7^ 


hn dvivtr'i \?\A l^f tyi-t. 


WW>^' % s'Wlvnjf^ i5 


SodJr/v?fv/ T krxjw <Hv9*h He wgj dw. 


nko m -mw 1Vc»ot "Wl hiv 


dyiVWs Ifcjyviic 3t> X ^dlicc 


bu+ h<^ ciK)v . A'nvwp^/ . f?^/rce 


khn^ fn''n9lly . 


X kf will ciet wk^t h iEservw fir 


5vclv 9- wd kt cW»6 . 


1 ’ ll ir/ylh ytfv Ojdm "fi hll yov 


9t?n)’{“ dtn'c ewy gffer div. smhncf clT 


J / 


tlUii'hA Wf/' 


p-^. _ 



All requirements of the task are fulfilled; ideas are effectively integrated 
and developed; a variety of simple and complex sentences used with 
some errors in usage 



Prototype Task GN2: Level B 



Advanced high (6) 



Dear Chris. 



You won’t believe what happened to me the other dayl Ai 



UjOU CHgW. 

m\l drtytr's licenii 



m 



y 



with rn\j ctrtytrs hctnie 

T /fAyg alMo^i ’for^oUin aho^rt 

HuL-h lyyiduciht hoiw Tia/Ci^ 






^ar/^nVecf tA/Aen T^a\AJ diyclnveri 
r.' r. - - J /. //).7 . ... A 



/i'ceti.s-e ftgA/n !One ^ 



iiet iti H\t y^htrt I 

»\/oril; . //g ^Aow^ol pnt dfivtrS, l!ce/i^e^ 



£jnd Tc,CimJ my owfi nClAte //. 



Called ci-f^nce^ Lui 'iha’t 



hB L 

I'liilt '^hief •f'Uol i/i/h 641 /ie ^eCirc/ 



H> 4 Hb j^ohc^ , 1 \A/c^ 



V^ry "friohi^fitd ^ h^tciu^d h€, Bvbi^ 
■for if oi 'ho h i£ ivct dt f />? ^ 



kufr\i .Thert vyO. %■ hti driver^, tic em e 



m f'^af lA/Oill^'f. colors fie iva^ 



A/re^fed v^ry 



iha'h 



If ^omthody And 4olc( mi he-fpro. 



^ut!^ tould ktr/ff rk t, 

Idh / hai^Q bdit Vi cl because I don 



kditvtcU 'because I ddn'-ftth'^yt U 
co'tnc'idtnce^ bu'f’ if did boPfien, I i^cknfed ’h 
WMdo you fbihk ahcmf ifr Id 
wolf for your U'Htr. jooup/as. 




All reqxiirements of the task are fulfilled; ideas are effectively integrated 
and developed; a variety of simple and complex sentences used with few 
errors 
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Prototype Task GN3 



Intermediate low (3) 



£ddo^: 

9 <eo4iiciUksto^ec<mme*ul 



Hexi Pe>iAon<4t^'^I^Bek. Hc Ot n^ Ce. 




a 6 >iUs 






y /A 



nfctJ:e 




/? 7 a^ 






i^y/th . a^ntos^ 



Ml 4 :^£uy ^ ^Ae/" £/^SHf 




..U Q2ac/yy/>/A 4^09^. 0^^ cenc^t^'^. 

^ISL- ^a/> 4 toc 7 a/ fitu^PLtiaxti . 

/?2i/^cu^ cc/zm'i^c/ ^S*7i 



Few task requirements are fulfilled; ideas are not integrated or 
developed; only simple sentences with errors 
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Prototype Task GN3 



Intermediate high (4) 

ubea^ QUi^ Sdiio^: 

9 wo4itl(llJisi»neoomme*ui ai^Uis 

mat Jc&gT 

hMyiCr, II 

\A)U(y\ - for 4 ^W l 

yUs Wife Ai^o) ^is. ^ rjill^Y^jn^ ItUM^ C4»^ X ^i»u) . 

VU- wfarhiAQ r\V 4W jg.VrauOkgjr^, 

"C W\ A y o>r fo\ V "Vt * 

RpfPr^ < agi> tntV9^ , ^‘Otrj 

Tkj "ilr\^-h Out If ///on Anj — Hl\% li^ 

cpfl/^g. /4 uJcL i 4o 

<nl^ 4-0 Hi*. We^ViOOr c J 

,A»VV>|/^* Sd m - i=J aajlS ill5 f^\iidYf.»^ 

u;/^ \40 mPOJdtIl . le>0-l> ^NAJtfA 

TL^poSS’.UU. ^^i)^ \f nLi->n A>vl(^r/Il . 

(•te- fe.V\n\i5;U < — Sd. — htJL gfl\^ ti6._ 

X^t \^ 4p y A-o 5w« Uor)^ . M l HoA Cvoj 

[V lAAflk^^ A'tfl&Mo ;a I UiOf ^ 

<jfd nt^i)4kjry^> fe>r^Pu-) gA«^_ 

UAf A\^ ^feJc ^^‘^4v^ ^v<>y’.V)Qj^.*- 

vu. \^ 1^ ^D/v> -VW -AtcVto^AVc aoi^ 

<hi^ ■lUI /It uiQi>4»^ -Vo ^ ^ 

CttA be 



Some requirements of the task are fulfilled; some ideas are integrated 
and developed while other ideas are not; predominantly simple 
sentences and some complex sentences with errors 

Note; Although this sample falls between minimal and partial task 
fulfillment, the language is best characterized by the third band of 
descriptors. Therefore, this sample is most representative of 
intermediate high writing. 
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Prototype Task GN3 



Advcuwed low (5) 



QUdf /^euf^ Cdito^: 

9 uumI( 1 like io' neaommenci 



ai>tUa 



next Pe^4oH>o|ftksUkek'. XJidh^ Ywfice Aou> A;< //f^ 
lye-ew. L»W' •f/towoi^e L ^yot^/ . ff>e fworr T’fe^l />/«? 
<=>t/<c/u h/ohj h*- }fPcaMf**prt.ii 0^ itn'x , 

A< u>g>S //■ lA^r ^ OCg*«»g/;/ <n>^ci ho 

}*4x 7pKaui'»«/g A-gtw»S l«<» T<0ti/i& ua/Zf/M kiUm^fet' 

’fo -PZ-gg “feow 4o» J»¥^9svy Au/ tfar. 

gyi fil /• A.h^ h^ck -ft, u/«< t/gy AakW ^t<y* 

Wh>fi.gi</ ^otf e^ucstfbh^ he r^</» 

P-f hl^lf SiHey^ work ^7K?iwo-/^_ A*5 tbt^rhu. 

•/» W wa/n. ht Oof bcfs /uc/ua ^ 

)v^ < VI "jh ^tUjippy’t J^v /suirf’u'is-tk . A/g voa< etxltt^ 

\A»ell i^v^irmorf ihj it/< lt!t coh^jpomu /ahltouz/i i\4. U»< l>ee.h a 

rlfc^jne-h', (, m»mw t//n^<‘; Hejo't' ir>ct^iU^ 



l;lci>J jf 



|rg<^to|r€ , 



Ae alfyto^'t dte,J haoiny som***i ke 

Vi'rtktv^ gytyf’Jki*^^ auJ Y€CO\/€YeJ . 



pmn t ivrrnpwcT a>*v-rftn|»^ <u.g> yecgi/grgq. 

^i’Ul (^lo'i of j»«l>rep**i IV* A.< hoJttj hiAf i\-P / 

\l)/^Y^\ho>)rA &*, i><6 f»h. ^ Ai» A’g 



)7<» c^twe fl. A^J hi. <1 Av'jw^fj 4-t ax-t’ 

Ao hs li~fe- 



«4 

;7" 



Most requirements of the task are fulfilled; ideas are usually integrated 
and developed; complex sentences usually attempted and advanced 
vocabulary used; some errors in form and usage 
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Prototype Task GN3 



Advanced high (6) 



Qttif, editor: 

9 UMuddldiet^^ecommewL /Wr 



o6>iUs 



hmI PM6s4tojftk&Week. 




Itbhery-A^ (X Jloia^JH.aXjt ^ 



^djuJUC A c/^tirw^ . CA^ s 

\Crw‘*jL ^ ***^*^ (f*^ <JcSt9t'^ 

:fe jtn/ig c.g>/^ ^ . Bk</1j 4^0 



A^iAs'fS tir»r,/n^r>X^ t c2. 

■^C.^>y*%-4UC \ji. l^t/>^ KmAS. 

ufr-M. <*>V< *^.»**^ — 

fti ^i'‘r«r. Vi^^oo ^ Lfnt^* A 

(C».y ^ t'/w^ 



^(^fk ? S» v>^ 

Si»^ /Xiu^ . fJ (M 



. A/^ f\^ lt/1 















*V^_U 



^ •JT 



jr>t. ^^LAyUf •'E'bLsir' Cxr>«^a.*«X>t jaIfCJ.9 a jA»<< Z 

*^ » C^*u<Y’ y cjKjuf %t 

tJTnrfc LtyrA. » A/o ij^ vthJSl 

"t^ “^d/'ja/i*^ So ^ tu^ 

^Ottn^wO**^ t^tj~ »»< ^yvC^t'Pifrf^L^ ^ Afcy UJmA 

H-* A/^ /VvW 9/X Oy^hr>.j;JljL^ . 



All reqmrements of the task are fulfilled; ideas are effectively integrated 
and developed; variety of simple and complex sentences used with few 



errors 
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Prototype Task GN4 



Retest Level A 



oi^ear l^eu/i 

^ wouid (ik^ io recomnuand ^ane ^eun^J^im ad tko next iParAon of ika lAJeeh, 

(/MJ 1sjSJ{/l (JV Ln, I3M- 

^UAJJkL U\ iU. "'-sfMjU i/L iSS^. 

Usju (aW^ m. flxAjjtiL iMuhejik 

^ I3S3 (jQj^ ui 0iAuuLi^ Jh>u£u , 

L^1 kM>/ ~ A^WxJiJUj-L JfUJjiL 

^--SiAAUJjU '^J^XjUUyJtzXJy, ‘ 

(XiuL Ul/333 a to/ (aS^aJl U^'^hjJLC^^M<i^ 

OuixL^ihUiUi, 



Few requirements of the task are fulfilled; ideas are not integrated or 
developed; although there are more than one or two simple sentences, 
most words and phrases used are directly copied from the prompt; 
frequent errors 

Note: Although this sample contains several simple sentences, most of 
the language is copied from the prompt. The test taker did not generate 
enough language for evaluation, therefore, retesting at the lower level is 
recommended. 
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Prototype Task GN4 



Intermediate low (3) 

^2)Uw* £JUon 

^ wou ti klt9 to roconunanJi ^Qtut^J Citn a6 ruuri j^rAon_ JlL WaaL. 

LOai. person, 3he ^et JoL . 6hr u;g4_- 

bort\ TAx in \ ^h^. 6^€ IA)Q*^ ! fX’ . 

ArY'wial I A. the. iJnlieJ ^tet-tes, 6)'c,<s -t t> 
<he to yD^rl lcal 

'j>yo9f'n n/\ do j'o A . — //V ^ 

f<AeJ()cQi cz't Cit/ fa/ri!l)/ chr\tc,. 

ifl lOf^l .^ht [jJo/l>e6 to ^^yeafs ~ I cQr\ J'tpu th 
^erv{re*> t, //^ /9f3 ^h-e ud^yit ’to 

lf\t>t(vcto/ 6-^ CDrt\rt)u.n'rt'j ^Irirt ^ aid cila£>s>e4. 

recomeiid ^^vTg/)g Je ujqJ^/jo 

pl-eo^ Aw a Qgfe. yf)^ny 

' ' 4,<indj C. u' n 

Some requirements of the task are fulfilled; ideas are loosely integrated 
and not developed; simple sentences only with frequent errors; some 
copying from prompt 
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Prototype Task GN4 



Intermediate high (4) 



jbtar Gtf HatM ^hhton 

la rotanunanA ^tutM a6 ikm txa xl j^Mon of 

SUt» ft P^ryx 1 .^ knfgfi^ [ja ttmg 

4-n Vu» V>vxiled | ^4-nVes Lso 1^35 — on£ L|eat 

IfiVer — Mf*f\^rnl osy!;isWpV 

f Yn^' .‘y/> O Cny nvn c» Hlgtg A l^COlS. 

jip^^ *5V^«a Ae<i'iA \q <.krli| — M . r . rWLPi - 

aV r.\-Vx^ Coraik^ — — Lio Q3J — 

Utg»Tf» \y\ V<>r>n^>*ny> > — Yrnttlft — %? . 1vl\Ce ^ 

l»^(ti^V«»er vjLig.rg* \jjQrK«v%<| \?.re — ffCR 

Ijj loeip 4kf. o«><>plp. 

\pivT-<\ rtW<l^V ike iink SSa£ — Skfc 

Lr<^^.^y^^rjgl- Cor \sija S^f.QTS j — \‘Vs30 \v^ S ViC . 

V\ere. a \y\«a.VfOg.W aC rr>raw <jy^\k| ^qt'A ^QgSfiS 



g»* 

Acj )oe 

r \(\<RPA 

i£2 feg- 



pet^ovn AvxA c>U& voong<»A VatA 

t«v;-Wt^c4nr qP Cfevni» Moni4i| f ffsV 

ii^ Vvoi^ V k££ sVfff\^ 

m Hve. V\e.uK<5 ^ Ql\g_ 

V\f>ui V^r4 . Skose SViQ\» VV^l^ 




i. 













Most requirements of the task are fulfilled; some ideas are integrated 
and developed; predominantly simple sentences with few complex 
sentences attempted; frequent errors 




Prototype Task GN4 



Advanced low (5) 

YJatvi £jiton 

J wouiJ. Aha to ncom monJ. f$ane ai ilia nud IPanott tka 

6)v.fc Ifiajl a. 'in. d 

XriA^C^iie M. "Tie <ir_n> , 

J-g.t4.rt. 

xG^ (X««-Ltic*P Sfe>JtcJ~ X.n. I?^6 ^ 

yU IA^caS -galLa , 

sWa. «• (JLnJtJihaJlji u>|f cu*ML,4Xe^. xjj - 






A^tea sUe. grriJLuJlJi -pt-oyt- g.x4,UUyd 7ru>^i^ 

, 5U. M cL^sli^ ot CX ^ 

C2Uc. , l3?f , ^ Wr:nJM 0>^ 0- 






V^o^ACA^tecA ^ i^fiocAA.— /^r^cAxaui Y<HCC^ $CACtfS 

^/AcACJg, k/^ VCt^-c^ (L<X^ {^Xh^aaJ^ C-fi c4»<, 



^|6a I^Ci^ ^ ^lyg e^uc6^c4^<L4$Zc AaA 

^iXccce^dLdi^ix.? "dEitt (Xail, /c<i^A^£a>HAjC^»i 



ilu, JjiJl l^c>t ;fcfr tesy KcOftAKJK^ 






So T IrecOTMkje^uS i^^/t guj nA>)C^ pg4r*di>n 

WeA^t . 



Sr^cjeAC^ ^ 



ru/t , 






■ S. JLa^ 



All requirements of the task are fulfilled; most ideas are integrated and 
developed; simple sentences with complex sentences usually attempted; 
some advanced vocabulary; some errors 

BEST COPY AVAILABLE 
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Prototype Task GN4 



Advanced high (6) 



:/r, 



^ibaar Gif How6 Glilor: 

^ wou UU. ahL WaoL 

Skt UJ«A £o'r>h. Px^tx^A^ eS.! . S^*. 

-ta -tLjt Uhi-Ud. . -^HtSr atfOt 

oLea^cLem/ 't* ^Ceco*-*^ a- . -o-o - ^Ls. 

■^t:eL>t^^->^- eU^Jt4\je! M^e4fifCAC O^ i'^'taL^ -6'tM4i-4rt^ 

S*L. "hufa 'avui.cC-»:^7C-^^ C o/t^/el^g^ 

!»'tOa'kjet*rK~ Mupj CV ^ ^ h^gCic^M- 

OJ^sd&.tct a.*^ Feu^A^l^ ^jatic^ (xiL s-r^ g~/-C ^"^It uvo-ricS. 

Oum. [7*^'**^ cut-xdrter job «o o ^ Cu^-trv^eto*^ 

/rj co»«x»fc»cfciU'^ •^r~^'t~ oJef uiL&r^ Sa/ at^uj. 

'to 'Ctgjx*>^ -T^n /wc LjtV~. Th- a^dicti'ti^tx^ ^ -^Le. c^ 

koreoM- Ay*t*/^co.^ yatjtU Sc<v',c€^ 



***" - 
^ h^^tC^C^CcL4ut^ Cf^ Atf6 y>z5g>^"p/w 

«/ 

^ , fcje«-e. 

S,t.cc<c-^ 



All reqmrements of the task are fulfilled; ideas are effectively integrated 
and developed; simple and complex sentences with few errors; advanced 
vocabulary used 
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Appendix E 



Rater Training Procedures for Scoring Generate Writing Tasks 
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Rater Training Procedures for Scoring Generate Writing Tasks 



This appendix contains recommended procedures for training raters 
to score Generate writing tasks. It is important to emphasize that the 
materials used in training should be directly related to the task being 
scored. This should be done to familiarize raters with the context in which 
a particular task is embedded, to provide training on representative 
samples, and to prompt discussions of task-related issues. Additionally, 
procedures for checking rater reliability must be established by the core 
development team to ensure that raters apply the rubric consistently. 
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Training Materials 

The following materials are needed to train raters to score Generate (GN) 
tasks: 

• Reading^vriting context 

The primary text, the secondary text (where appropriate), the writing 
prompt, and any reading comprehension questions associated with the 
primary text 

• Appropriate level scoiing rubric 

See discussion of rubrics under Scoring GN Tasks above. 

• Elxamples of complex sentences and verb forms 

A list of sentences that illustrate complex sentence types (e.g., sentences 
with relative clauses, adverbials, conditionals) and complex verb forms 
(e.g., present perfect, past continuous, passive voice) 

Sentences should be related to the writing task in that they could 
plausibly be part of a response. Following are some example sentences 
from protot 3 rpe writing task GN2: 



Complex sentence types 

The man who stole my wallet came into the store. 

When he came into the store I called the police. 

Although I felt sorry for him, I called the police. 

If the boy had realized that I was Douglas Cleaver, he would never have 
tried to buy beer with my license. 

Complex verb forms 

I have worked/have been working at Wally’s for three years. 

I was working behind the counter when the man came in. 

My wallet was stolen last March. 

He must have stolen the wallet from my truck. 



Four sets of writing samples 

Representative writing samples selected from responses to be scored 
Set 1: Anchor papers 

One writing sample per proficiency level presented in ascending order 
(e.g., BL, BH, IL, IH) 

Each sample is clearly representative of the proficiency level. 

Set 2: Norming papers 

One writing sample per proficiency level presented in random order 
Each sample is clearly representative of the proficiency level. 

Sets 3 and 4: Norming papers 

Four writing samples not necessarily from each proficiency level 

Some of the samples illustrate issues or problems that raters may 
encounter (e.g., cop 3 dng, integration of ideas, task specific issues). 

Note: Additional samples should be prepared in case raters need more 
practice. 

Sample score sheet 

A score sheet such as the one shown below that allows for up to three 
ratings 

Booklet number 

ist Read 

Rater ID 

Score 

2nd Read 

Rater ID 

Score 

3*‘d Read (if necessary) 

Rater ID 

Score 

Note: The sample score sheet provided is smaller than the original. 
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Training Procedures 



Prior to the training session 

• Select raters 

Raters should be teachers or counselors familiar with the adult student 
population and trained in ESL instruction. 

* Prepare training materials 

Trainers must read through several writing samples to select papers 
appropriate for use in the training sets described above. If there is more 
than one trainer, the trainers should be in agreement on score 
assignments for all samples included in the sets. 

Trainers should also note any issues or problems associated with a 
given task that may not be addressed in the rubric. 



Training session 



• Present rubric 



Explain how the rubric is organized (see discussion above) and review 
definitions of task fulfillment and content development as well as 
examples of complex verb tenses and structures. 



It is important to remind raters that writing samples are being scored 
holistically and that not all samples will exactly match the descriptors 
provided. Raters should be told that they will need to determine the 
descriptors that best fit the sample being scored as quickly as possible. If 
they have any questions during the training about application of the 
rubric, they should immediately ask a trainer. 



• Discuss writing task 

Provide raters with copies of the appropriate materials including the 
primary text, secondary text, writing prompt, and any reading 
comprehension questions associated with the primary text. 

Raters must be familiar with the writing task as well as the context in 
which the task is embedded to determine the degree to which the test 
taker incorporates information from these sources in his or her 
response. 



Review Set 1 (Anchor papers) 



xl- _ X Ci _ j 









i.eii rabei's tiiut oet i CGiibaiiiS representative samples oi cacn proficiency 
level in ascending order. Draw attention to language characteristics 
and degree of task fulfillment as they apply to each sample presented. 
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Rate Set 2 (Norming papers) 

Tell raters that Set 2 also contains representative samples that 
correspond to each proficiency level, but in this set they are presented in 
random order. Give raters a few minutes to assign scores to each 
sample in the set. When raters have finished, they should share the 
scores they assigned and any discrepancies should be discussed. 

Note: Because this set contains one sample from each level, raters 

should be able to make decisions easily. 

Rate Set 3 (Norming papers) 

Tell raters that Set 3 contains four samples, but that there is not 
necessarily a sample that corresponds to each level. Allow raters 
enough time to score each sample in the set. Ask raters to share the 
scores they assigned and lead a discussion to resolve any issues or 
discrepancies before continuing to the next set of samples. 

Rate Set 4 (Norming papers) 

Follow the same procedvu*es as Set 3 above. 

By the time they finish Set 4, raters should feel comfortable with the 
rubric and its application. If not, it is best to practice scoring additional 
samples. 

Discuss reliability issues 

Discuss reliability issues and procedures for monitoring both inter- and 
intra-rater reliability. 

A procedure for checking rater reliability must be established by the core 
development team to ensure that raters apply the rubric consistently 
when scoring writing samples. 

Review scoring procedures 

Explain that each sample is scored independently by two raters using 
numbers that correspond to Model Standards proficiency levels. (See 
Scoring GN Tasks in Writing Task Specifications.) 

Refer to a sample score sheet and explain that the first rater indicates 
the test booklet number, rater ID (number or initials), and score 
assignment in the spaces provided. 

To ensure that the second rater scores each sample without seeing the 
score of the first rater, the first rater folds the score sheet back such that 
first rater information cannot be seen (first rater information face down; 
second rater information face up) and attaches the score sheet the 
writing sample with a paper clip. 

When the second rater receives the sample, he or she indicates rater ID 
and score assignment in the spaces provided. After scoring the sample, 
the second rater checks to see if the two scores are within one point. If 
the two scores are not within one point, a third rater scores the sample 
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providing the same information as the first and second raters. If a third 
rater is not available, the two raters discuss the sample and come to 
agreement about the score assignment. 
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